Microbenchmarking of luajit-based server (again: this time against lighttpd)

The other day inbetween the meetings I've ported my toy event loop experiment to use ljsyscall library.

Here are the results of running of ab -n 100000 -c 1000. First let's make a baseline:

lighttpd

# ab -n 100000 -c 1000 http://localhost:80/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Finished 100000 requests


Server Software:        lighttpd/1.4.19
Server Hostname:        localhost
Server Port:            80

Document Path:          /
Document Length:        10 bytes

Concurrency Level:      1000
Time taken for tests:   7.854757 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      24331590 bytes
HTML transferred:       1001300 bytes
Requests per second:    12731.14 [#/sec] (mean)
Time per request:       78.548 [ms] (mean)
Time per request:       0.079 [ms] (mean, across all concurrent requests)
Transfer rate:          3025.05 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   39 281.0     15    3023
Processing:     5   19  16.5     21     740
Waiting:        3   14  15.9     14     734
Total:          9   58 287.6     38    3746

Percentage of the requests served within a certain time (ms)
  50%     38
  66%     40
  75%     42
  80%     43
  90%     45
  95%     47
  98%     50
  99%     94
 100%   3746 (longest request)
#

luajit

Now let's try it on the primitive event loop.

# ab -n 100000 -c 1000 http://localhost:12345/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            12345

Document Path:          /
Document Length:        13 bytes

Concurrency Level:      1000
Time taken for tests:   8.232656 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      5503355 bytes
HTML transferred:       1300793 bytes
Requests per second:    12146.75 [#/sec] (mean)
Time per request:       82.327 [ms] (mean)
Time per request:       0.082 [ms] (mean, across all concurrent requests)
Transfer rate:          652.77 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   44 336.4      6    3016
Processing:     4   10  13.9     10     766
Waiting:        3    8  13.9      7     762
Total:         10   55 341.0     16    3767

Percentage of the requests served within a certain time (ms)
  50%     16
  66%     17
  75%     17
  80%     18
  90%     20
  95%     22
  98%     28
  99%   3018
 100%   3767 (longest request)

Upon multiple runs the numbers vary slightly, of course, but they stay within the same ballpark. I think this shows luajit is a very viable platform for server development.

----

Update:

curious cat as I am, I've added a HTTP parser that is made by a yet-unpublished-and-inefficient-and-incomplete patch for ragel that generates Lua state machines (and a ragel code from an earlier post about http parser.

ab -n 100000 -c 100 http://localhost:12345/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            12345

Document Path:          /
Document Length:        19 bytes

Concurrency Level:      100
Time taken for tests:   21.980 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      6100000 bytes
HTML transferred:       1900000 bytes
Requests per second:    4549.62 [#/sec] (mean)
Time per request:       21.980 [ms] (mean)
Time per request:       0.220 [ms] (mean, across all concurrent requests)
Transfer rate:          271.02 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       3
Processing:     1   22   3.1     21      30
Waiting:        1   22   3.1     21      30
Total:          3   22   3.1     21      30

Percentage of the requests served within a certain time (ms)
  50%     21
  66%     25
  75%     25
  80%     25
  90%     25
  95%     26
  98%     27
  99%     27
 100%     30 (longest request)

This is not stellar, but I did not see yet where the bottlenecks are and if I can speed it up. Either way the 4.5K is still not too bad.

Update2:

with concurrency of 1000, I notice connection resets... After fixing up the mixing SIGPIPE handler of SIG_IGN, they still happen, apparently. Further debugging pending...