A little background
- heavy network transfers;
- slow startup.
Introducing the Binary AST
Producing a Binary AST file will require a build step and we hope that, in time, build tools such as WebPack or Babel will be able to produce Binary AST files, hence making switching to Binary AST as simple as passing a flag to the build chains already used by many JS developers.
I plan to detail the Binary AST, our benchmarks and our current status it in future blog posts. For the moment, let me just mention that early experiments suggest that we can both obtain very good source compression and considerable parsing speedups.
- it is designed to make parsing much faster;
- according to early experiments, we beat gzip or brotli by a large margin.
Note that our main objective is to make parsing faster, so in the future, if we need to choose between file size and parsing speed, we are most likely to pick faster parsing. Also, the compression formats used internally may change.
The tool traditionally used by web developers to decrease the size of JS files is the minifier, such as UglifyJS or Google’s Closure Compiler.
Minifiers typically remove unused whitespace and comments, rewrite variable names to shorten then, and use a number of other transformations to make the program shorter.
While these tools are definitely useful, they have two main shortcomings:
- they do not attempt to make parsing faster – indeed, we have witnessed a number of cases in which minification accidentally makes parsing slower;
By opposition, the Binary AST transformation:
- is designed to make parsing faster;
- maintains the source code in such a manner that it can be easily decoded and read, with all variable names, etc.
Of course, obfuscation and Binary AST transformation can be combined for applications that do not wish to keep the source code readable.
Now, exploring this would definitely be an interesting work, so if anybody wants to prove us wrong, by all means, please do it :)
These are problems that we address with Binary AST.
…we improved caching?
Additional technologies have been discussed to let browsers prefetch and precompile JS code to bytecode.
These technologies are definitely worth investigating and would also help with some of the scenarios for which we are developing Binary AST – each technology improving the other. In particular, the better resource-efficiency of Binary AST would thus help limit the resource waste when such technologies are misused, while also improving the cases in which these techniques cannot be used at all.
…we used an existing JS bytecode?
So, one could imagine browser vendors exposing their bytecode and letting all JS applications ship bytecode. This, however, sounds like a pretty bad idea, for several reasons.
The second affects JS developers. Having several bytecodes would mean maintaining and shipping several binaries – possibly several dozens if you want to fine-time optimizations to successive versions of each browser’s bytecode. To make things worse, these bytecodes will have different semantics, leading to JS code compiled with different semantics. While this is in the realm of the possible – after all, mobile and native developers do this all the time – this would be a clear regression upon the current JS landscape.
…we had a standard JS bytecode?
Just to be clear: I have heard people regretting that such a format did not exist but I am not aware of anybody actively working on this.
Also, whether such a bytecode would actually help code size and performance, remains to be demonstrated.
…we just made the parser faster?
Wouldn’t it be nice if we could just make the parser faster? Unfortunately, while JS parsers have improved considerably, we are long past the point of diminishing returns.
Let me quote a few steps that simply cannot be skipped or made infinitely efficient:
- dealing with exotic encodings, Unicode byte order marks and other niceties;
- finding out if this
/character is a division operator, the start of a comment or a regular expression;
- finding out if this
(character starts an expression, a list of arguments for a function call, a list of arguments for an arrow function, …;
- finding out where this string (respectively string template, array, function, …) stops, which depends on all the disambiguation issues, …;
- finding out whether this
let adeclaration is valid or whether it collides with another
const adeclaration – which may actually appear later in the source code;
- upon encountering a use of
eval, determine which of the 4 semantics of
- determining how truly local local variables are;
In either case, the VM needs to perform an expensive pre-parse step that can often backfire into being slower than regular parsing, typically when applied to minified code.
We are posting this blog entry early because we want you, web developers, tooling developers to be in the loop as early as possible. So far, the feedback we have gathered from both groups is pretty good, and we are looking forward to working closely with both communities.
We have completed an early prototype for benchmarking purposes (so, not really usable) and are working on an advanced prototype, both for the tooling and for Firefox, but we are still a few months away from something useful.
I will try and post more details in a few weeks time.
For more reading: