Non puoi eseguirlo in quel modo...l'oggetto deve essere caricato prima di eseguire lo script...quindi:

Codice PHP:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Untitled Document</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">

</head>

<body>

<div id="layer" style="position:absolute; left:0px; top:0px; width: 100%; overflow: auto; z-index:1; background-color: #FFCCCC; layer-background-color: #FFCCCC; border: 1px none #000000;">
BLA, BLA,</div>

<script type="text/javascript">
document.getElementById("layer").style.height = (window.innerHeight - 300)+"px";
</script>

</body>
</html> 
prova un po'