A friend from Red Hat sent me an email asking about Kiwi TCMS performance so I did an experiment to establish a baseline. For API requests I got 7.5 req/sec or 130 msec/req which is 1.5x slower than GitHub!
to measure that. The script takes the first 250 test cases from our test suite
and on every execution creates a new TestPlan (1 API request), then creates
new test cases (250 requests), adds cases to test plan (250 requests),
creates new product build (1 API request), creates new TestRun (1 API request),
adds test cases to test run (250 requests) and finally updates the statuses
A total of 1003 API requests are sent to Kiwi TCMS every time you start this script! An example is available at TR #567!
On localhost, running the development server (
./manage.py runserver) with an
SQLite database I got:
$ time python perf-script real 2m6.450s user 0m1.069s sys 0m0.331s $ time python perf-script real 2m7.472s user 0m1.057s sys 0m0.342s $ time python perf-script real 2m9.368s user 0m1.072s sys 0m0.351s $ time python perf-script real 2m9.197s user 0m1.050s sys 0m0.353s
This measures at 120 msec/req or 7.85 req/sec!
demo.kiwitcms.org is running on an
instance (via docker-compose) with the default
No extra settings or changes. I used the same computer over a WiFi
connection and a pretty standard home-speed Internet connection. Times are:
$ time python perf-script real 2m18.983s user 0m1.175s sys 0m0.095s $ time python perf-script real 2m25.937s user 0m1.156s sys 0m0.108s $ time python perf-script real 2m24.120s user 0m1.102s sys 0m0.098s $ time python perf-script real 2m21.521s user 0m1.154s sys 0m0.091s
This measures at 140 sec/req or 7.05 req/sec!
Note: since I am using Python 3.6 I had to modify the file
/opt/rh/rh-python36/root/lib64/python3.6/ssl.py to read:
# Used by http.client if no context is explicitly passed. _create_default_https_context = _create_unverified_context # this disables HTTPS cert validation
The issue has been reported in RHBZ #1643454