PROFILE

NodeJS Coldfusion Benchmark

NodeJS vs. Coldfusion Benchmark

I started out as a front-end developer, my main skills lies in javascript. It was an obvious step for me to get involved with NodeJS, and I have loved every bit of it. On the other hand, at my day job, we use CFML to power our backend logic. So I decided to make some tests between how well does CFML performs against NodeJS and how do they compare. I prepared a couple of simple examples that goes from making a simple 'hello world' to querying a mysql db and displaying the results.

The tests were done on a Digital Ocean instance of 8gb of ram, 4 core processor and having MySQL served locally. The tests were performed under the same conditions, using CentOS with NGINX as proxy server, Railo v4.2 for CFML and v0.10.28 for NodeJS. Note: I used the cluster module for NodeJS in order to use all CPU cores for the tests.

For benchmarking, Apache HTTP Server Benchmarking Tool was selected for measuring the performance of the HTTP web servers (originally designed to test the Apache HTTP Server, though, it is generic enough to test any web server).

What I'm focusing on these tests is how many requests can CFML and NodeJS handle per second. The two commands used specify 1000 requests with 10 and 100 concurrent connections.

ab -r -n 1000 -c 10 [url]  
ab -r -n 1000 -c 100 [url]  

Hello World Test

<!--- hello.cfm --->  
Hello World  
// hello.js
// Simple Server Initialization for NodeJS
// Responding with 'Hello World'
var sys = require('sys'),  
    http = require('http'),
    cluster = require('cluster'),
    numCPUs = require('os').cpus().length;

if(cluster.isMaster) {  
    // Fork workers.
    for (var i = 0; i < numCPUs; i++) {
        cluster.fork();
    }

    cluster.on('exit', function(worker, code, signal) {
        console.log('worker ' + worker.process.pid + ' died');
    });
} else {
    var server = http.createServer(function(req, res) {
        res.writeHead(200, {'Content-Type': 'text/html'});
        res.write('Hello World!');
        res.end();
    }).listen(3000);
}

**Spoiler Alert**: This is the only test that CFML won over NodeJS. This is a simple 'hello world', nothing else. I really thought Node would beat the heck out of CF (because CF loads all the wiring for a server instance and Node only creates the server and responds with a "hello world"), surprisingly for me, this was the test that CF won. We can see, from the first chart, that Coldfusion did better than Node, but the difference is really small, just 0.25% better than NodeJS. However, on the second chart, I increased to 100 concurrent connections and Node improved, it was 13.18% better than CF. Here are both charts:




Loop Test

<!--- loop.cfm --->  
<cfoutput>  
    <cfloop from="0" to="1000" index="step">
        #step# <br>
    </cfloop>
</cfoutput>  
// loop.js
// Simple Server Initialization for NodeJS
// Creates a loop of 1000 and writes to browser
var sys = require('sys'),  
    http = require('http'),
    cluster = require('cluster'),
    numCPUs = require('os').cpus().length;

if(cluster.isMaster) {  
    // Fork workers.
    for (var i = 0; i < numCPUs; i++) {
        cluster.fork();
    }

    cluster.on('exit', function(worker, code, signal) {
        console.log('worker ' + worker.process.pid + ' died');
    });
} else {
    var server = http.createServer(function(req, res) {
        var body = '';
        for (var i = 0; i <= 1000; i++) {
            body += i+'<br>'
        };
        res.writeHead(200, {'Content-Type': 'text/html'});
        res.write(body);
        res.end();
    }).listen(3000);
}

For the loop test, I created a loop of 1000 iterations, each time printing HTML. With 10 concurrent connections, we see that NodeJS did 25% better than CF. For the 100 concurrent connections, NodeJS was a staggering 98.96% better! Another point for NodeJS!




File Read and JSON parsing Test

<!--- json.cfm --->  
<cffile  
    action = "read"
    file = "#getDirectoryFromPath(getCurrentTemplatePath())#random.json"
    variable = "data"
    >

<cfoutput>  
    <cfloop array="#DeserializeJSON(data)#" index="row">
        <h2>#row.company#</h2>
        <p>#row.about#</p>
        <img src="#row.picture#" alt="">
    </cfloop>
</cfoutput>  
// json.js
// Simple Server Initialization for NodeJS
// Reads a file(json) and parse it
var sys = require('sys'),  
    http = require('http'),
    fs = require('fs'),
    cluster = require('cluster'),
    numCPUs = require('os').cpus().length;

if(cluster.isMaster) {  
    // Fork workers.
    for (var i = 0; i < numCPUs; i++) {
        cluster.fork();
    }

    cluster.on('exit', function(worker, code, signal) {
        console.log('worker ' + worker.process.pid + ' died');
    });
} else {
    var server = http.createServer(function(req, res) {
        var body = '';
        fs.readFile('./random.json',function(err, data){
            var data = JSON.parse(data);
            for (var i = 0; i < data.length; i++) {
                body += '<h2>'+data[i].company+'</h2>';
                body += '<p>'+data[i].about+'</p>';
                body += '<img src="'+data[i].picture+'" alt="" />';
            };
            res.writeHead(200, {'Content-Type': 'text/html'});
            res.write(body);
            res.end();
        });
    }).listen(3000);
}

In this test, I used a random json structure saved in a file. Then Coldfusion and NodeJS, had to read that file, parse the JSON and show the results in the browser. As you can see, the first test (10 concurrent connections), Node won by 9.74% and with 100 concurrent connections, the difference was by 25.79%.




MySQL Query Test

<!--- mysql.cfm --->  
<cfquery datasource="classicmodels" name="test">  
    SELECT *
    FROM customers
</cfquery>

<table>  
    <tr>
        <th>Customer #</th>
        <th>Customer Name</th>
        <th>Phone</th>
        <th>City</th>
        <th>Country</th>
        <th>Credit Limit</th>
    </tr>
    <cfloop query="#test#">
        <cfoutput>
            <tr>
                <td>#customerNumber#</td>
                <td>#customerName#</td>
                <td>#phone#</td>
                <td>#city#</td>
                <td>#country#</td>
                <td>#creditLimit#</td>
            </tr>
        </cfoutput>
    </cfloop>
</table>  
// mysql.js
// Simple Server Initialization for NodeJS
// Query a DB and show results
var http = require('http'),  
    mysql = require('mysql'),
    cluster = require('cluster'),
    numCPUs = require('os').cpus().length,
    connection = mysql.createConnection({
        host     : 'localhost',
        user     : 'root',
        password : '*******',
    });

if(cluster.isMaster) {  
    // Fork workers.
    for (var i = 0; i < numCPUs; i++) {
        cluster.fork();
    }

    cluster.on('exit', function(worker, code, signal) {
        console.log('worker ' + worker.process.pid + ' died');
    });
} else {
    var server = http.createServer(function(req, res) {
        var body = '';
        connection.query('SELECT * FROM classicmodels.customers', function(err, rows) {
            if(err) console.log(err);

            body+= '<table><tr>';
            body+= '<th>Customer #</th>';
            body+= '<th>Customer Name</th>';
            body+= '<th>Phone</th>';
            body+= '<th>City</th>';
            body+= '<th>Country</th>';
            body+= '<th>Credit limit</th></tr>';
            for (var i = 0; i < rows.length; i++) {
                var row = rows[i];
                body+= '<tr><td>'+row.customerNumber+'</td>';
                body+= '<td>'+row.customerName+'</td>';
                body+= '<td>'+row.phone+'</td>';
                body+= '<td>'+row.city+'</td>';
                body+= '<td>'+row.country+'</td>';
                body+= '<td>'+row.creditLimit+'</td></tr>';
            };
            body+= '</table>';
            res.writeHead(200, {'Content-Type': 'text/html'});
            res.write(body);
            res.end();
        })
    }).listen(3000);


    process.on('SIGINT', function(){
        connection.destroy()
        server.close();
    });
}

This was the test I was most interested in, querying a database and showing the results in the browser, which is the core of a dynamic website. At the initial 10c test, I saw that Node won by 0.39% only... I was kind of disappointed, as I thought Node would beat the crap out of CFML here. Then I try using 50 concurrent connections and Node won by only 4.74%... it had improved, but not as I expected/wanted. But when I tried the 100 concurrent connections, Node improved significantly to 51% better than CF. Here are the results for 10c and 100c:




Conclusion

Based on these results, NodeJS vs CFML... the winner is... NodeJS! Coldfusion only won on one category (A simple Hello World) and it wasn't by much --just 0.25% better--.

With a small amount of concurrent connections, both CF and Node did ok, but it was until a higher amount of concurrent connections hit the server that we can really see NodeJS as the shinning winner, performing significantly better than Coldfusion.

If anyone thinks that the tests are not accurate, please let me know in the comments so I can improve them. No caching or framework was used so that we can get a more accurate result. Here are the links to github of the code used for these tests as well as the screenshots of each test. CFML Link , NodeJS Link.

comments powered by Disqus