workadventure/benchmark
2020-10-21 15:15:07 +02:00
..
.gitignore Adding first protobuf message exchange 2020-09-17 17:14:47 +02:00
benchmark_multi_core.sh Switching test loading from Artillery to home-grown test 2020-09-18 17:01:45 +02:00
index.ts Increasing node memory to 4GB 2020-10-21 15:15:07 +02:00
package-lock.json Improving benchmark 2020-10-21 14:55:18 +02:00
package.json Migrating to uWS 2020-09-28 18:52:54 +02:00
README.md Moved benchmark to its own directory and added multicore testing + a README 2020-09-11 09:56:05 +02:00
yarn.lock Migrating to uWS 2020-09-28 18:52:54 +02:00

Load testing

Load testing is performed with Artillery.

Install:

cd benchmark
npm install

Running the tests (on one core):

cd benchmark
npm run start

You can adapt the socketio-load-test.yaml file to increase/decrease load.

Default settings are:

  phases:
    - duration: 20
      arrivalRate: 2

which means: during 20 seconds, 2 users will be added every second (peaking at 40 simultaneous users).

Important: don't go above 40 simultaneous users for Artillery, otherwise, it is Artillery that will fail to run the tests properly. To know, simply run "top". The "node" process for Artillery should never reach 100%.

Reports are generated in artillery_output.html.

Multicore tests

You will want to test with Artillery running on multiple cores.

You can use

./artillery_multi_core.sh

This will trigger 4 Artillery instances in parallel.

Beware, the report generated is generated for only one instance.

How to test, what to track?

While testing, you can check:

  • CPU load of WorkAdventure API node process (it should not reach 100%)
  • Get metrics at the end of the run: http://api.workadventure.localhost/metrics
    In particular, look for:
    # HELP nodejs_eventloop_lag_max_seconds The maximum recorded event loop delay.
    # TYPE nodejs_eventloop_lag_max_seconds gauge
    nodejs_eventloop_lag_max_seconds 23.991418879
    
    This is the maximum time it took Node to process an event (you need to restart node after each test to reset this counter)
  • Generate a profiling using "node --prof" by switching the command in docker-compose.yaml:
        #command: yarn dev
        command: yarn run profile
    
    Read https://nodejs.org/en/docs/guides/simple-profiling/ on how to generate a profile.