Title: A line-to-line conversion from Node.js to Node.cpp

Author: Bob Schmidt

Date: 05 February 2020 18:57:27 +00:00 or Wed, 05 February 2020 18:57:27 +00:00

Summary: Dmytro Ivanchykhin, Sergey Ignatchenko and Maxim Blashchuk show how we can get a 5x improvement in speed.

Body: 

Disclaimer: as usual, the opinions within this article are those of ‘No Bugs’ Hare, and do not necessarily coincide with the opinions of the translators and Overload editors; also, please keep in mind that translation difficulties from Lapine (like those described in [Loganberry04]) might have prevented an exact translation. In addition, the translators and Overload expressly disclaim all responsibility from any action or inaction resulting from reading this article.

It is pretty well known that, computation-wise, C++ code is substantially faster than Node.js (even with all the efforts spent on the v8 engine); for details we can refer to [Debian], where for different calculation-oriented algorithms the results were in favour of C++, with differences in wall-clock time ranging from 1.2x to 6x.

However, most of the benchmarks out there (including [Debian]) are concentrating on pure computations. This means that differences in the ability of different frameworks to handle requests (which forms a basis for handling real-world interactive loads) are not addressed. This article aims to start covering this gap.

Node.cpp

At this point, we want to compare good old Node.js with our own new kid on the block, which we named Node.cpp.

The idea behind Node.cpp is to make a framework which will allow us to write C++ code in Node.js style while benefiting from C++ goodies (including significantly improved performance). Moreover,

It should be possible to take existing Node.js code and convert it into Node.cpp with line-to-line correspondence between the two.¹

As of now, Node.cpp is still very much in its infancy, but we have already managed to write enough code to run some benchmark tests Let’s take a look at the code of the http ‘echo’ server which is along the lines of the sample from [Ostinelli11] (see Listing 1).

//Node.js
http.createServer(function(request, response) {
  if ( request.method == "GET" 
       || request.method == "HEAD" ) {
    response.writeHead(200, 
      {"Content-Type":"text/xml"});
    var urlObj = url.parse(request.url, true);
    var value = urlObj.query["value"];
    if (value == ''){
      response.end("no value specified");
    } else {
      response.end("" + value + "");
    }
  } else {
    response.writeHead(405, 
      "Method Not Allowed");
    response.end();
  }
}).listen(2000);

//Node.cpp
srv = net::createHttpServer<ServerType>(
   [](net::IncomingHttpMessageAtServer& request,
      net::HttpServerResponse& response){
      if ( request.getMethod() == "GET" 
           || request.getMethod() == "HEAD" ) {
        response.writeHead(200, 
          {{"Content-Type", "text/xml"}});
        auto queryValues =
          Url::parseUrlQueryString(
          request.getUrl() );
        auto& value = queryValues["value"];
        if (value.toStr() == ""){
       response.end("no value specified");
     } else {
       response.end( value.toStr() );
     }
      } else {
        response.writeHead( 405, 
          "Method Not Allowed");
        response.end();
      }
  });
srv->listen(2000, "0.0.0.0", 5000);
            

As we can see, there is a direct correspondence between Node.JS code and Node.cpp code; sure, there are quirks related to the nature of C++ (and some more due to still-missing APIs in Node.cpp – which will be fixed before release), but overall the whole thing looks similar enough

to enable manual but more or less mechanistic rewriting from Node.js into Node.cpp

Sure, after rewriting into Node.cpp the code will be still a bit more verbose, but what is important is that such a rewrite should be feasible without any changes to the essence of the Node.js code.

What’s the point?

Ok, we can see that it IS possible to convert some rudimentary Node.js code into Node.cpp without breaking the structure of existing Node.js code. But what is the point of going through such an exercise?

In the not so distant future, we’re planning to add some ultra-useful features to Node.cpp – such as deterministic recording/replay (which in turn will allow production post-mortem debugging (sic!)); however, for the time being, we’ll concentrate on one advantage of Node.cpp – namely, on performance.

Test setup

When testing our Node.cpp against Node.js, our plan was to:

• Exclude testing of computations (there are too many computations out there, and they are addressed by other benchmarks such as [Debian])
• Have a test which is as close to real-world conditions as possible

To achieve this, we used httperf along the lines described in [Ostinelli11], with some changes intended (a) to reflect the real world better (most importantly, we ran our tests between two separate boxes), and (b) restricting the programs under test to use one single core (multi-core tests using cluster module are coming, but they’re not a part of this particular article).

Hardware

Unlike [Ostinelli11], we ran our client and server on two different boxes:

• Server

  HP DL380eG8 (12xLFF), CPU: 2x Intel Xeon E5-2420 (6 cores, 12 threads, 15M Cache, 1.90 GHz, 7.20 GT/s Intel® QPI), RAM 32 GB, Disks 4x3TB SATA, Network card: 10GE UTP card

  Overall, our test server is pretty much a typical workhorse 2S server, which has tended to dominate data centres for at least for last 20 years.

• Client

  Dell R630, CPU: 2x Intel Xeon E5-2630v4, 128G RAM, Disks 2x480GB SSD

  Honestly, client hardware doesn’t matter much and is mentioned here merely for the sake of completeness.

Client and server boxes were interconnected directly via a 10Gbit switch.

Software

Both client and server boxes were running stock Ubuntu 19.10 (Eoan).

On the client, we ran httperf patched to use an increased number of file descriptors as discussed in [Stackoverflow].

We used this command line for httperf:

  httperf --timeout=5 --client=0/1
  --server=10.32.36.3 --port=2000  --rate=XXX
  --send-buffer=4096 --recv-buffer=16384
  --num-conns=8000 --num-calls=70

where the (session) rate parameter went from 100 to 2000 in steps of 100.

On the server, we ran either Node.js, which is available for eoan (10.15.2), or the open-source code for [node.cpp] compiled with Clang 9. App-level code used for Node.js and Node.cpp is shown in Listing 1 above. Another test we ran (just as a sanity check and to put things into perspective) was the raw performance of the single-core nginx serving static files (which are requested by the same httperf); in a sense, nginx results represent ‘The Holy Grail’ of http dynamic processing, something we can try to reach.

Results

The results of our testing are shown in Figure 1.

Figure 1

Here, along the lines of [Ostinelli11], the ‘desired’ response rate is calculated as rate × num-calls (as specified in the command line for httperf), and the ‘real’ response rate is the actual response rate as measured by httperf.

As we can see, with a lower load, all the servers behave similarly until the capacity of a particular server is reached; but after that limit is reached, however much we increase the load, the response rate doesn’t really improve and stays more or less stable.

As such, we can conclude (at least within the limitations of the current test setup), that Node.js can handle a maximum of 13,000 responses/sec per core, while Node.cpp can handle around 70,000 responses/sec (that’s over a 5× advantage performance-wise(!!)). Static-serving nginx, as expected, goes well above both dynamic handlers (at 100,000 responses/sec), but we have to note that (i) a 30% difference between Node.cpp and nginx is not THAT bad, and (ii) we will try to bring node.cpp MUCH closer to the performance of static-serving nginx.

Important notes about the results:

• All the results are currently for a single core only. Tests for multiple cores (using cluster API) are coming soon.
• Node.cpp has a 5× performance advantage over Node.js

  5× is a Damn Lot™. It is also interesting to note that other frameworks competing with Node.js (such as Erlang/Elixir and golang) seem to have performance in the same ballpark as Node.js, so Node.cpp can become a competitive advantage of the Node.* ecosystem (more on this below). We’re planning to come to this question in our next article, after running multi-core tests adding Erlang, Elixir, and golang to the mix.

• The big fat question is where Node.js (as well as competing frameworks) manage to lose that 5× performance improvement (we DO realize that it is not Node.cpp which performs well, it is rather Node.js which performs poorly). More investigation would be needed to find out why it is slower.
• Node.cpp is merely 30% behind the statically-serving nginx.

  This number looks even more impressive as Node.cpp didn’t even start with any optimizations of its code (it is not pessimized, but that’s it); in particular, Node.cpp is currently using generic poll() rather than Linux-specific epoll() – and is still merely 30% behind The Holy Grail of http performance.

• Our tests were intentionally run for an as-small-as-feasible piece of code; we did NOT try to measure the performance of the computations within the language (this is covered by [Debian] and other benchmarks); instead, we tried to test the maximum performance of the respective frameworks.

  This means that for this particular test where we’re implementing an echo http server, using C++ plug-ins for Node.js wouldn’t bring any observable benefit (what would we do within a C++ plug-in to implement echo? Copy a string from input to output?)

• Within these tests, the performance of Node.cpp with runtime memory safety enabled was not that much different from its performance with runtime memory safety checks turned off (NB: this applies ONLY to this particular test; in general, such results depend heavily on the memory structures used, and may vary greatly).

Great! When can we start conversion?

As noted above, the whole point of Node.cpp is to allow conversion of (those 5% of performance-critical Nodes which warrant such an effort) from Node.js into Node.cpp.

The only tiny problem on the way is that Node.cpp is still in its early infancy, and is not ready (yet) for production use. In particular, the set of supported APIs is still extremely limited, and package management is not there yet. If you want to help, please feel free to contribute (or contact the authors); we feel that Node.cpp is a project with wonderful prospects that can help make Node.* ecosystem an indisputable leader at least performance-wise (and a common point of view is that Node loses performance-wise both to Erlang/Elixir and to Golang [Christensen16] [Peabody] [Stressgrid20]²).

Conclusions and future work

Over the course of this article, we took a very rudimentary Node.js http server, and converted it more or less line-by-line into Node.cpp. Then we ran a bunch of http tests to compare performance of both versions, and found that the Node.cpp server outperforms the Node.js one by a factor of 5×. We feel that this opens up wonderful opportunities and are going to continue our work to enable high-performance Node.* programming for those Nodes where performance is critical.

Acknowledgement

Cartoon by Sergey Gordeev from Gordeev Animation Graphics, Prague.

References

[Christensen16] Jack Christensen (2016) ‘Websocket Shootout: Clojure, C++, Elixir, Go, NodeJS, and Ruby’ at: https://hashrocket.com/blog/posts/websocket-shootout, posted 1 September 2016

[Debian] The Computer Language Benchmarks Game: ‘Node js versus C++ g++ fastest programs’ https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/node-gpp.html

[Loganberry04] David ‘Loganberry’, Frithaes! – an Introduction to Colloquial Lapine!, http://bitsnbobstones.watershipdown.org/lapine/overview.html

[Ostinelli11] Roberto Ostinelli (2011) ‘A comparison between Misultin, Mochiweb, Cowboy, NodeJS and Tornadoweb’ at: http://www.ostinelli.net/a-comparison-between-misultin-mochiweb-cowboy-nodejs-and-tornadoweb/

[node.cpp] node.cpp, https://github.com/node-dot-cpp/node.cpp

[Peabody] Brad Peabody ‘Server-side I/O Performance: Node vs. PHP vs. Java vs. Go’ at: https://www.toptal.com/back-end/server-side-io-performance-node-php-java-go

[Stackoverflow] lawnmowerlatte ‘Changing the file descriptor size in httperf’ at: https://stackoverflow.com/a/16449853/4947867

[Stressgrid20] Stressgrid (2020) ‘Benchmarking Go vs Node vs Elixir’ at: https://stressgrid.com/blog/benchmarking_go_vs_node_vs_elixir/, posted 6 January 2020

1. At this point, we’re talking about a manual rewrite; whether automated conversion will be possible, and how efficient it will be, is currently beyond the scope of this article.
2. We do know that at least some of these tests are unfair to Node.js (by ignoring cluster module), and that Node is generally rather competitive to Erlang/Elixir and Golang, but 5x performance improvement would clearly blow all the competition out of the water.

Dmytro Ivanchykhin has 10+ years of development experience, and has a strong mathematical background (in the past, he taught maths at NDSU in the United States).

Sergey Ignatchenko has 15+ years of industry experience, including being a co-architect of a stock exchange, and the sole architect of a game with 400K simultaneous players. He currently holds the position of Security Researcher.

Maxim Blashchuk has substantial development experience, most of it with embedded programming. Recently he joined a team performing research on low-level C++ libraries providing properties such as determinism and memory safety.

Notes: 

More fields may be available via dynamicdata ..