Kiwi TCMS is the winner at OpenAwards'19 category Best Tech Community! Big thanks to the jury, our contributors and core-team and the larger open source and quality assurance communities who voted for us and supported the project during all of those years.
This award is the best present we could get to mark the 10th anniversary of the project. More news of how we are progressing with current roadmap will follow soon in a separate blog post.
Thank you & happy testing!
We're happy to announce Kiwi TCMS version 6.10! This is a small security and improvement update. You can explore everything at https://public.tenant.kiwitcms.org!
Supported upgrade paths:
5.3 (or older) -> 5.3.1 5.3.1 (or newer) -> 6.0.1 6.0.1 -> 6.1 6.1 -> 6.1.1 6.1.1 -> 6.2 (or newer)
Docker images:
kiwitcms/kiwi latest bbb581d60ed1 1.005 GB kiwitcms/kiwi 6.2 7870085ad415 957.6 MB kiwitcms/kiwi 6.1.1 49fa42ddfe4d 955.7 MB kiwitcms/kiwi 6.1 b559123d25b0 970.2 MB kiwitcms/kiwi 6.0.1 87b24d94197d 970.1 MB kiwitcms/kiwi 5.3.1 a420465852be 976.8 MB
Kiwi TCMS is exhibitor at OpenExpo Europe on June 20th in Madrid. We will be hosting an info booth and 2 technical presentations delivered by Anton Sankov and Alex Todorov.
Kiwi TCMS is also the leading finalist in the Best Tech Community category at Open Awards 2019!
Use OE19SPEAKERINVITE 100% discount code! Get your ticket here!
If you are using Kiwi TCMS as a Docker container then:
cd Kiwi/ git pull docker-compose down docker pull kiwitcms/kiwi docker pull centos/mariadb docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate
Don't forget to backup before upgrade!
WARNING: kiwitcms/kiwi:latest and docker-compose.yml will always point to the latest available version! If you have to upgrade in steps, e.g. between several intermediate releases, you have to modify the above workflow:
# starting from an older Kiwi TCMS version docker-compose down docker pull kiwitcms/kiwi:<next_upgrade_version> edit docker-compose.yml to use kiwitcms/kiwi:<next_upgrade_version> docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate # repeat until you have reached latest
Happy testing!
Thanks to you, our community supporters, Anton Sankov and Alex Todorov took the lead at OpenExpo 2019 CfP votes. We need your help one more time. Our team has submitted participation in 'Best Tech Community' and 'Best Success Story' categories.
Unfortunately our submission into 'Best Success Story' has been pulled down! We used that category to share the story from a dead open source project into a thriving open source community with lots of users and contributors and to highlight some of our milestones. Here's the short version:
Please help us gain more recognition:
Thanks you & happy testing!
On Tuesday I hosted my pylint workshop during the regular Django Bulgaria meetup. This edition was the first which was practice based.
Attendance numbers were low but participation was very good. We managed to create 4 new checkers for Kiwi TCMS:
Many thanks to all contributors. These new checkers have discovered quite a few new issues with Kiwi TCMS so this is an area which our team is going to improve.
Those who missed the workshop will be able to catch up one of the next editions:
Happy testing!
We're happy to announce Kiwi TCMS version 6.9! This is a small improvement and bug-fix update which introduces our first telemetry report: testing breakdown. You can explore everything at https://public.tenant.kiwitcms.org!
Supported upgrade paths:
5.3 (or older) -> 5.3.1 5.3.1 (or newer) -> 6.0.1 6.0.1 -> 6.1 6.1 -> 6.1.1 6.1.1 -> 6.2 (or newer)
Docker images:
kiwitcms/kiwi latest a01eaeddf213 1.001 GB kiwitcms/kiwi 6.2 7870085ad415 957.6 MB kiwitcms/kiwi 6.1.1 49fa42ddfe4d 955.7 MB kiwitcms/kiwi 6.1 b559123d25b0 970.2 MB kiwitcms/kiwi 6.0.1 87b24d94197d 970.1 MB kiwitcms/kiwi 5.3.1 a420465852be 976.8 MB
Kiwi TCMS is exhibitor at OpenExpo Europe on June 20th in Madrid. We will be hosting an info booth and 2 technical presentations delivered by Anton Sankov and Alex Todorov.
Next week we are going to announce 100% discount tickets for our guests at the conference. If you are interested in coming subscribe to our newsletter and don't miss the announcement!
If you are using Kiwi TCMS as a Docker container then:
cd Kiwi/ git pull docker-compose down docker pull kiwitcms/kiwi docker pull centos/mariadb docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate
Don't forget to backup before upgrade!
WARNING: kiwitcms/kiwi:latest and docker-compose.yml will always point to the latest available version! If you have to upgrade in steps, e.g. between several intermediate releases, you have to modify the above workflow:
# starting from an older Kiwi TCMS version docker-compose down docker pull kiwitcms/kiwi:<next_upgrade_version> edit docker-compose.yml to use kiwitcms/kiwi:<next_upgrade_version> docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate # repeat until you have reached latest
Happy testing!
We're happy to announce Kiwi TCMS version 6.8! This is a small improvement and bug-fix update. You can explore everything at https://public.tenant.kiwitcms.org!
Supported upgrade paths:
5.3 (or older) -> 5.3.1 5.3.1 (or newer) -> 6.0.1 6.0.1 -> 6.1 6.1 -> 6.1.1 6.1.1 -> 6.2 (or newer)
Docker images:
kiwitcms/kiwi latest fca095f95475 994.8 MB kiwitcms/kiwi 6.2 7870085ad415 957.6 MB kiwitcms/kiwi 6.1.1 49fa42ddfe4d 955.7 MB kiwitcms/kiwi 6.1 b559123d25b0 970.2 MB kiwitcms/kiwi 6.0.1 87b24d94197d 970.1 MB kiwitcms/kiwi 5.3.1 a420465852be 976.8 MB
During the past week we have silently launched Kiwi TCMS on GitHub Marketplace. Free plans will be redirected to public.tenant.kiwitcms.org while paid plans will be hosted under *.tenant.kiwitcms.org sub-domains!
Our team is still working on all requirements for GitHub integration and we will issue a formal statement when ready!
If you are using Kiwi TCMS as a Docker container then:
cd Kiwi/ git pull docker-compose down docker pull kiwitcms/kiwi docker pull centos/mariadb docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate
Don't forget to backup before upgrade!
WARNING: kiwitcms/kiwi:latest and docker-compose.yml will always point to the latest available version! If you have to upgrade in steps, e.g. between several intermediate releases, you have to modify the above workflow:
# starting from an older Kiwi TCMS version docker-compose down docker pull kiwitcms/kiwi:<next_upgrade_version> edit docker-compose.yml to use kiwitcms/kiwi:<next_upgrade_version> docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate # repeat until you have reached latest
Happy testing!
We're happy to announce Kiwi TCMS version 6.7! This is a small improvement and bug-fix update. You can explore everything at https://public.tenant.kiwitcms.org!
Supported upgrade paths:
5.3 (or older) -> 5.3.1 5.3.1 (or newer) -> 6.0.1 6.0.1 -> 6.1 6.1 -> 6.1.1 6.1.1 -> 6.2 (or newer)
Docker images:
kiwitcms/kiwi latest 702a78976de7 993.5 MB kiwitcms/kiwi 6.2 7870085ad415 957.6 MB kiwitcms/kiwi 6.1.1 49fa42ddfe4d 955.7 MB kiwitcms/kiwi 6.1 b559123d25b0 970.2 MB kiwitcms/kiwi 6.0.1 87b24d94197d 970.1 MB kiwitcms/kiwi 5.3.1 a420465852be 976.8 MB
Our team is happy to announce the availability of kiwitcms-junit-plugin. This is a native JUnit 5 plugin which will read the results from your automated test suite and send them back to Kiwi TCMS.
Version 1.0.3 and the latest changes in master branch are still to be considered Beta quality. They are available for early preview and feedback.
Many thanks to Aneta Petkova who is the main contributor behind this plugin!
If you are using Kiwi TCMS as a Docker container then:
cd Kiwi/ git pull docker-compose down docker pull kiwitcms/kiwi docker pull centos/mariadb docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate
Don't forget to backup before upgrade!
WARNING: kiwitcms/kiwi:latest and docker-compose.yml will always point to the latest available version! If you have to upgrade in steps, e.g. between several intermediate releases, you have to modify the above workflow:
# starting from an older Kiwi TCMS version docker-compose down docker pull kiwitcms/kiwi:<next_upgrade_version> edit docker-compose.yml to use kiwitcms/kiwi:<next_upgrade_version> docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate # repeat until you have reached latest
Happy testing!
We're happy to announce Kiwi TCMS version 6.6! This is a medium severity security update, improvement and bug-fix update. You can explore everything at https://public.tenant.kiwitcms.org!
Supported upgrade paths:
5.3 (or older) -> 5.3.1 5.3.1 (or newer) -> 6.0.1 6.0.1 -> 6.1 6.1 -> 6.1.1 6.1.1 -> 6.2 (or newer)
Docker images:
kiwitcms/kiwi latest c4734f98ca37 971.3 MB kiwitcms/kiwi 6.2 7870085ad415 957.6 MB kiwitcms/kiwi 6.1.1 49fa42ddfe4d 955.7 MB kiwitcms/kiwi 6.1 b559123d25b0 970.2 MB kiwitcms/kiwi 6.0.1 87b24d94197d 970.1 MB kiwitcms/kiwi 5.3.1 a420465852be 976.8 MB
Populate missing TestCase.text history. In version 6.5 the TestCase model was updated to store the text into a single field called text instead of 4 separate fields. During that migration historical records were updated to have the new text field but values were not properly assigned.
The "effect" of this is that in TestCaseRun records you were not able to see the actual text b/c it was None.
This change ammends 0006_merge_text_field_into_testcase_model for installations which have not yet migrated to 6.5 or later. We also provide the data-only migration 0009_populate_missing_text_history which will inspect the current state of the DB and copy the text to the last historical record.
Remove legacy reports. Closes Issue #657
Remove "Save & Continue" functionality from TestCase edit page
Renamed API methods:
These methods work with URL links, which can be added or removed to test case runs.
If you are using Kiwi TCMS as a Docker container then:
cd Kiwi/ git pull docker-compose down docker pull kiwitcms/kiwi docker pull centos/mariadb docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate
Don't forget to backup before upgrade!
WARNING: kiwitcms/kiwi:latest and docker-compose.yml will always point to the latest available version! If you have to upgrade in steps, e.g. between several intermediate releases, you have to modify the above workflow:
# starting from an older Kiwi TCMS version docker-compose down docker pull kiwitcms/kiwi:<next_upgrade_version> edit docker-compose.yml to use kiwitcms/kiwi:<next_upgrade_version> docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate # repeat until you have reached latest
Happy testing!
We are happy to announce that Anton Sankov and Alex Todorov are currently taking the lead at OpenExpo Europe's CfP votes!
Going to OpenExpo will be huge boost for Kiwi TCMS so please help us make this happen! Voting is open until March 17th 2019! You can cast your vote via Facebook login but remember to confirm your email address!
Thank you & happy testing!
As we've promised Kiwi TCMS is starting work on improving the legacy reports functionality. After analyzing your feedback in Issue #657 here's what we came up with. Note: screenshots below are mockups.
Inside Kiwi TCMS the term "test case" is used to refer to specific scenario of testing steps. A "test run" is the collection of multiple "test cases" which will be executed against a particular product build and environment.
The specific results are kept into what is now called "test case run". This will
be renamed to "test execution" (internally TestExecution class) to make it more
clear and easier to distinguish among other artifacts.
As a webUI user I want to easily distinguish between PASSED, FAILED and other statuses when looking at charts and graphical visualizations.
This feature request comes from the TestRun progress bar. Other charts should use the same 4 status categories and color coding standards:
pf-black-600pf-green-400pf-red-100pf-orange-400. This includes other statuses which are not PASSED, FAILED and IDLE.Here the percent value will continue to mean percent completed, aka non-IDLE.
Implementation wise the TestCaseRunStatus class defines methods for the actual icons to be used.
The same can be done for colors and this can be used directly in the HTML templates.
For color definitions see Status Palette.
As a tester I want to have a nice looking TestRun execution report which I can present to my managers, stakeholders and/or customers. The page needs to be printer friendly for “Print to PDF” or print on paper!
Kiwi TCMS already has this feature. Clicking the [Report] button
in the progress widget at TR-625 presents
TestRun execution information in a simplified format found at
TR-625/Report.
In order to improve the current implementation:
testruns.views.TestRunReportView and associated templates,
JavaScript and CSS because they will not be needed anymoreAs a manager I want to know what the work-load/performance of individual teams and engineers is. I can use this information either for performance review or for capacity planning of future work.
A new page will be available under TELEMETRY -> Management -> Performance which will:
The backend will query TestExecution.objects and apply the selected
filters and grouping logic!
The returned data is the count of how many tests this person/team was able to execute.
Further refinement: Aggregate count by statuses (executed vs. non-executed or PASSED vs. FAILED vs. OTHER). May be represented as a stacked bar chart if grouping of series is supported.
As a manager I want to know how much time it took to complete previous testing activities. I can use this information for predicting future capacity.
New page under TELEMETRY -> Management -> Time tracking.
Visually this will be a line chart with one line for each user/group which have been selected. It will be very good if the existing chart libraries allow for a stacked line chart or a stacked area chart here!
As a test lead I want to know approximately when testing is expected to complete.
This feature will utilize time-tracking data from previous executions and display the approximated calculation in the web UI. The most obvious place for this is the TestRun progress widget (under the progress bar) or the Test Run page (near the Started at/Finished at fields).
Example text:
Started at: YYYY-MM-DD 10:38
Status: Running
Duration: 1.2 / 3 hrs
Estimated completion at: 13:30
Unknown: where does baseline duration come from? Either from the most recent TestRun or an average from all of the previous Test Runs. Note that cumulative TestRun duration is calculated between Start/Finish events while individual TestExecution records may sum up to a different value! This needs to be clarified.
As a tester and/or manager I want to be able to discover which TestCase(s) are the most common source of problems, that is they fail all the time or from time to time. This may indicate poor test design or bad practices when developing the product. This is also known as test stability or flaky tests!
New page under TELEMETRY -> Testing -> Flaky tests
TestExecution.objectsAs a tester and/or manager I want to see a breakdown of tests so that I have an idea how the existing tests for the product are distributed!
New page under TELEMETRY -> Testing -> Breakdown
Shows combined bar/line charts with information about test case priorities, categories, automation status and confirmed status.
As a test engineer I want to be able to quickly see health information about individual test case. This can help me visually locate TCs that need improvement so I can edit them or will help me visually judge the past history of the TC I am looking at.
This depends on the previous feature. Possible places where health status can be shown:
As a test manager I want to see information if Product builds are getting better or worse. In other words what is the trend in TestExecution number and status!
This is going to be a new page at TELEMETRY -> Testing -> Execution trends
TestExecution.objects (stacked area chart) with color codes for statusYou will be able to select TestRuns and/or weeks for detailed comparison.
As a test manager I want to see detailed information about testing status for specific test run(s) or a calendar period. I also want to be able to compare the results between various weeks and/or test runs.
This is a continuation of Execution trends telemetry but can be used on its own as well. It will be accessible from TELEMETRY -> Testing -> Status matrix
Filtering options are the same. Results are displayed in a table format where columns are the filtered (or previously selected) TestRun objects or calendar weeks. The rows are the names of individual test cases that have been executed. If a case is missing the cell will be empty.
Cells contain color coded information about status, timestamp of execution, execution duration, links to the TestExecution object, latest comment in case of failure.
Links to bugs will be shown in each cell with a summarized bug table below the charts.
This is complimentary to the TestRun status matrix feature! While searching for TestRuns in SEARCH -> Test Runs the user will be able to select several rows and proceed to compare them as described above.
Happy testing!
Have you ever wanted to be part of an open source team? Have you ever wanted to contribute back the open source community ? Have you ever wanted to see your code used by thousands of people ?
If yes now you have the opportunity! Read on to learn how you can help Kiwi TCMS and how our team can help you.
It is fine not to have any experience at all! You will compensate with commitment and hard work. Initially you are going to work on refactoring, cleaning up pylint errors, removing duplicate code and other issues reported by CodeClimate.
By doing this you will have the opportunity to learn git, Python, Django, some CSS, JavaScript and Patternfly HTML of course. We are going to provide you with all the learning materials plus help and guidance from existing team members.
Everyone on the team has gone though the same training procedure and grueling tasks and so will you! Once you can demonstrate progress and learn the ropes you will continue working on more complicated tasks.
So you have some experience already, you've probably contributed code before and are now looking for more green stripes on your GitHub profile. We've got you covered!
There are many areas to choose from: issue tracker integration, GitHub integration, GitLab integration, external API library, Kiwi TCMS plugins written in Python and customized pylint linters! This is going to be where you get your hands dirty and show your strengths. Our team is here to help if necessary but we expect you to show progress by yourself.
A challenge for you will be to review pull requests from other contributors and be patient with less experienced contributors and team members. This is an excellent opportunity to work on your people skills as well.
Kiwi TCMS is primarily looking for Java developers who will own our test automation plugins. Currently we have a plugin for JUnit 5 and TestNG is in planning. Maybe there will be a plugin for Jenkins as well. You are going to own these components and work solely on them. Unless you decide to learn Python and Django that would be a very easy job!
.NET, PHP, Ruby, JavaScript ? We don't have a lot of code written in these languages but you can help change this. The main thing we'd like you to know (or become familiar with) are the internals of popular test automation frameworks for these languages and how to create plugins for them.
You are going to test a lot! You are going to write test automation a lot! Ideally you already have a medium level of experience in the software testing field and want to improve your coding skills and/or get more experience into a different application domain. We also have Linux and Docker in the mix, just for fun!
Your responsibility will be to design test scenarios for various features (new or existing), write test automation scripts and help improve overall test coverage and quality of Kiwi TCMS. You will also check-in on non-Python developers and help them with test design when necessary.
There are other things that can be tested as well, for example Kiwi TCMS performance and scalability. Here you will have to get down to the nitty-gritty stuff and do some profiling to pin-point where the root cause of the problem is.
We've got Coverity scan and Snyk automatically inspecting our code base. We do have some other tools as well and we know they can never be enough.
You will be responsible for triaging the numerous issues being reported by these tools and help us decide if they are a real threat or a false positive. For example Coverity reports hundreds of issues mostly coming from our Python and Node.js dependency stack. We haven't had the time to classify them and work with upstream communities to fix them thus the majority of your contributions will be outside of the Kiwi TCMS code base.
Your main job is going to be creating beautiful images for our website, blog posts and promotional material. All the images we use are licensed under Creative Commons which we then modify with the specific Kiwi TCMS look and feel. This is not going to change, your work will remain under a permissive license!
You will be directly responsible for driving more traffic to our website, interpreting Google Analytics metrics and coming up with creative ideas how to boost Kiwi TCMS popularity. This means, but not limited to blog posts, collaborations with other projects and/or bloggers, professional magazines, etc. You will also be in charge of events and conferences that we go to! Whenever possible you will be coming with us as well!
A challenge for you will be to learn some technical jargon and learn more about the software testing profession and software testers in general!
You will sharpen your skills! You will use Kiwi TCMS as a platform to improve your career. You will experience the gratification of our community of users.
This blog is the medium where you can share tips and tricks and technical articles about interesting features in Kiwi TCMS. If you'd rather have your personal blog working on Kiwi TCMS will give you lots of topics to write about.
We go to conferences and meetups too. If public speaking is your thing you will have plenty of topics to talk about. We can also help you deliver your first presentation! Everyone on the team has done it!
Our existing team will help you learn and we will help you grow. Our personal time is the most expensive item we can offer to you! In return we expect you to fulfill your commitments and when you promise something will be done you will make sure it is done!
You can figure this out yourself.
Happy testing!
In release notes for v6.5 we announced several plugins which will fetch test names and execution results from your automated test suite.
Plugins can be controlled via environment variables which will affect how test results are recorded in the Kiwi TCMS database! This blog post is an introduction how that works and what you can do with it! For this purpose I will be using the plugin-demo repository, which simulates real development work.
Full documentation and list of available plugins is available in chapter Automation Frameworks Plugins!
The default behavior is always to create a new TestRun if controlling variables are not overridden! Product name, version and build will receive default values if tests are running in Travis CI or Jenkins. For example Travis Build #2 for commit d455fb4 creates TR-12 and TP-10!
Product=kiwitcms/plugin-demo
Version=d455fb42fb7c2aedadfd5f66de7d131109c03350
Build=2
Next we convert the README file from Markdown to reStructuredText which triggers Travis Build #3 for commit 418b80b. This build again creates a new TestRun and new TestPlan for it. Respectively TR-14 and TP-12!
Product=kiwitcms/plugin-demo
Version=418b80b3bbb65a799f695dc59d488c76f560fa2b
Build=3
Important: we can see that version is different which will affect how artifacts are organized in Kiwi TCMS, possibly affect how you will report status to stakeholders!
Let's say the QA team has decided that all test results must be reported under the
same TestPlan. This means version must be the same between various builds in Travis CI!
To control this we export TCMS_PRODUCT_VERSION=master in CI before executing
the TAP plugin! Checkout the commit on GitHub to see how it is done!
This triggers Travis Build #4 for commit e484e59. Because this is the first time where Version == master the plugin creates TP-14 and reports the results under TR-16.
Product=kiwitcms/plugin-demo
Version=master
Build=4
Right after that I realized we can make this configuration a bit more generic
because our team is planning to support multiple versions of the product and
development will be done in separate branches!
Travis Build #5 for
commit f1f2878
still ends up with Version == master because we are still working on the master
branch! That is to say, the product is in active mode of development.
Results are reported in TR-18 which is again part of TP-14.
Product=kiwitcms/plugin-demo
Version=master
Build=5
Travis Build #6 for commit df6153b adds the new functionality README badges and reports test results in TR-20 which is again part of TP-14.
While giving status reports back to stakeholders and developers the information that we have in the TestRun is Build number! This follows the numbering scheme in Travis CI (or Jenkins job number) and isn't very useful.
Let's define TCMS_BUILD to be the first 7 characters of the commit hash! When QA
tells devel that something isn't working and redirects them to the TestRun they can
immediately use the Build information and git checkout the offending variant of the product
for investigation.
This results in Travis Build #7 for commit bf75d0a, TR-22, TP-14.
Product=kiwitcms/plugin-demo
Version=master
Build=bf75d0a
There are many reasons you may want to do this:
To do so I've created TR-24 beforehand and
configured TCMS_RUN_ID=24 in my CI environment. TR-24 also contains
TC-57: Verify we can report results from several plugins into the same TR.
this was created and added via the web interface.
Note: mixing additional test cases can be done either before or after automation results are reported with the plugin!
Important: when reporting results to an existing TestRun Kiwi TCMS plugins
don't care in which TestPlan this TR is! In theory it is possible to report
the results for Product=kiwitcms/plugin-demo into any TP/TR pair! However we
don't want to do this crazy thing and instead I've created TR-24 inside the already
existing TP-14!
Note: because I don't know what is the git commit beforehand I've configured
TR-24 with Build=unspecified. If I wanted I could update this with the correct value
once I know the commit hash for the related changes I am testing.
Important: the plugin-demo repository uses both kiwitcms-tap-plugin and kiwitcms-junit.xml-plugin at the same time! They differ in the way test names are compiled! Both are reported in TR-24. See TC-57 text for information how to distinguish between the two.
See Travis Build #8 for commit 85ad939, TR-24, TP-14.
Product=kiwitcms/plugin-demo
Version=master
Build=unspecified
Also check-out comments in TR executions to see when and who had executed the test case.
So far there have been some tests which were failing (although Travis reports PASS) so
I decided to fix them.
Travis Build #9 for
commit a25b384
is still configured with TCMS_RUN_ID=24. This means results will be reported in
TR-24, effectively overwriting previous results.
Check-out Change Log under each individual execution and you will see several
time stamps when status was updated! In other words, as long as TCMS_RUN_ID is
pointing to an existing TestRun this TR will keep the results of the last test suite execution!
Travis Build #10 for
commit c4f1ae5
removes TCMS_RUN_ID! Results are reported in
TR-25,
TP-14.
Product=kiwitcms/plugin-demo
Version=master
Build=c4f1ae5
Our team has decided to make the first LTS release of the product. We branch out into
lts-v0 branch in git and cut the first NVR. This results in
Travis Build #11 for
commit 9f1ef71
TR-27,
TP-16.
Product=kiwitcms/plugin-demo
Version=lts-v0
Build=9f1ef71
Because this is the first time we are running tests for this product version we end up with the newly created TP-16!
Note: see how version was populated with the correct value for the git branch!
This is because my CI config uses TCMS_PRODUCT_VERSION=$TRAVIS_BRANCH and no further
changes were required! I made this change back in Travis Build #5 anticipating what
will come in the future!
The product is now live and customers have reported critical bugs for it: URLs for the badges in README are wrong! I fix those and add more tests of course, see: Travis Build #12 for commit 2d72754, TR-29 for TP-16.
Product=kiwitcms/plugin-demo
Version=lts-v0
Build=2d72754
TR-29 contains the new TC-58!
In the lts-v0 branch customers have identified a serious issue. It doesn't exist on master
but the test is still valid so I cherry-pick it! In git you can backport or forwardport very
easily. Regardless of the direction Kiwi TCMS plugins will record the test results for you.
See Travis Build #13 for commit 31ae5b3, TR-31 for TP-14.
We can see that TC-58, which was originally implemented on the lts-v0 branch
is now present!
This is a very basic example of how you can organize collection and reporting for
your automation test suite results with Kiwi TCMS. The links here only refer to
artifacts created by kiwitcms-tap-plugin while in the DB we keep others as well.
There are feature requests for more ENV variables which will allow you to control
TestPlan creation and child/parent relationship between test plans. See
https://github.com/kiwitcms/tap-plugin/issues and vote with a :+1: reaction
to help us plan for these features.
Kiwi TCMS automation framework plugins are nothing more than result parsers which talk back to a database. It is up to you to decide how to organize the collection of test results and how to report on them later, when the need arises.
Future installments in this post series will take a look at workflows with feature branches and pull requests and discuss possible organization scenarios. You are welcome to share your ideas in the comments below.
Happy testing!
We're happy to announce Kiwi TCMS version 6.5.3! This is a security, improvement and bug-fix update that includes new versions of Django, includes several database migrations and fixes several bugs. You can explore everything at https://public.tenant.kiwitcms.org!
Supported upgrade paths:
5.3 (or older) -> 5.3.1 5.3.1 (or newer) -> 6.0.1 6.0.1 -> 6.1 6.1 -> 6.1.1 6.1.1 -> 6.2 (or newer)
Docker images:
kiwitcms/kiwi latest b9355cf85833 1.039 GB kiwitcms/kiwi 6.2 7870085ad415 957.6 MB kiwitcms/kiwi 6.1.1 49fa42ddfe4d 955.7 MB kiwitcms/kiwi 6.1 b559123d25b0 970.2 MB kiwitcms/kiwi 6.0.1 87b24d94197d 970.1 MB kiwitcms/kiwi 5.3.1 a420465852be 976.8 MB
If you are using Kiwi TCMS as a Docker container then:
cd Kiwi/ git pull docker-compose down docker pull kiwitcms/kiwi docker pull centos/mariadb docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate
Don't forget to backup before upgrade!
WARNING: kiwitcms/kiwi:latest and docker-compose.yml will always point to the latest available version! If you have to upgrade in steps, e.g. between several intermediate releases, you have to modify the above workflow:
# starting from an older Kiwi TCMS version docker-compose down docker pull kiwitcms/kiwi:<next_upgrade_version> edit docker-compose.yml to use kiwitcms/kiwi:<next_upgrade_version> docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate # repeat until you have reached latest
Happy testing!
We're happy to announce Kiwi TCMS version 6.5! This is a minor security, improvement and bug-fix update that includes new versions of Django and other dependencies, removes some database fields, includes backend API updates and fixes several bugs. You can explore everything at https://public.tenant.kiwitcms.org!
Together with this release we announce:
Both of these are implemented in Python and should work on standard TAP and junit.xml files generated by various tools!
Additionally 3 more plugins are currently under development by contributors:
Supported upgrade paths:
5.3 (or older) -> 5.3.1 5.3.1 (or newer) -> 6.0.1 6.0.1 -> 6.1 6.1 -> 6.1.1 6.1.1 -> 6.2 (or newer)
Docker images:
kiwitcms/kiwi latest 7ca4fda602fc 1.002 GB kiwitcms/kiwi 6.2 7870085ad415 957.6 MB kiwitcms/kiwi 6.1.1 49fa42ddfe4d 955.7 MB kiwitcms/kiwi 6.1 b559123d25b0 970.2 MB kiwitcms/kiwi 6.0.1 87b24d94197d 970.1 MB kiwitcms/kiwi 5.3.1 a420465852be 976.8 MB
Known issues: on our demo installation we have observed that permission labels were skewed after applying migrations. The symptom is that labels for removed models are still available, labels for some models may have been removed from groups/users or there could be permission labels appearing twice in the database.
This may affect only existing installations, new installations do not have this problem!
We are not certain what caused this but a quick fix is to remove all permissions from the default Tester group and re-add them again!
If you are using Kiwi TCMS as a Docker container then:
cd Kiwi/ git pull docker-compose down docker pull kiwitcms/kiwi docker pull centos/mariadb docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate
Don't forget to backup before upgrade!
WARNING: kiwitcms/kiwi:latest and docker-compose.yml will always point to the latest available version! If you have to upgrade in steps, e.g. between several intermediate releases, you have to modify the above workflow:
# starting from an older Kiwi TCMS version docker-compose down docker pull kiwitcms/kiwi:<next_upgrade_version> edit docker-compose.yml to use kiwitcms/kiwi:<next_upgrade_version> docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate # repeat until you have reached latest
Happy testing!
Hello testers, Kiwi TCMS is going to FOSDEM this year. This is where you can find us:
Shout out with #KiwiTCMS on Twitter if you can't find us in the crowdWe would like to meet with all of you and talk about test management and test process organization. In case you are stuck for crazy ideas checkout our project mission for inspiration.
We are turning 10 years old and we have presents for you! You will have to perform a small challenge and you can get your hands(errr, feet) on a pair of these:
Here's what else you can do to help us:
Star button on our GitHub repositoryHappy testing!
Hello testers, Kiwi TCMS has taken on a brave new mission! We would like to transform the testing process by making it more organized, transparent & accountable for everyone on your team. Our goal is to improve engineering productivity and participation in testing. The following blog post outlines how we would like to achieve this and what goals we put before ourselves for this year.
Last year we took on the challenge to bring a legacy code base up to modern coding standard. We did not complete that effort but made very good progress along the way. This is not a small task and that's why our team will continue with it this year.
postToURL() and jQ.ajax() on the front-endThere are 59 templates remaining to be converted to a modern look and feel. Along with them comes more refactoring and even redesign of the existing pages and the workflow around them. Together with refactoring this will make Kiwi TCMS easier to use and also to maintain.
We are planning to remove the existing reports feature because they are not well designed. We will re-implement existing functionality that our community finds useful, add new types of reports (incl. nicer graphics and UI) and make it possible for the reporting sub-system to be more easily extendable.
Phase out is planned to begin after 1st March 2019! Until then we are looking for your feedback. Please comment in Issue #657!
These will make it easier to collect results from automated test suites into Kiwi TCMS for later analysis. Instead of creating scripts that parse the results and talk to our API you will only have to install an additional package in your test environment and configure the test runner to use it! Automation test results will then appear inside Kiwi TCMS.
If you would like to use such functionality leave your vote inside GitHub issues! In case you would like to write a test-runner plugin you can find the specification here.
Question: Does Kiwi TCMS integrate with JIRA?
Answer: Well, it does. How exactly do you want to integrate?
... silence ...
The following dialog happens every time someone asks me about bug-tracker integration, especially with JIRA. The thing is integration is a specified set of behavior which may or may not be desired in a particular team. As of now Kiwi TCMS is able to open a URL to your bug-tracker with predefined field values, add comments to bug reports and report a simple summary of bugs inside a TestRun.
We recognize this may not be enough and together with the community we really need to define what bug tracker integration means! The broader domain of application lifecycle management tools (of which TCMS is a sub-set) have an integrated bug tracking system. We can add something like this and save you the trouble of using JIRA, however many teams have already invested in integrating their infrastructure or just like other tools. For example we love GitHub issues and our team regularly makes public reports about issues that we find internally!
Developers have their GitHub PR flow and if they have done the job of having unit tests then they will merge only when things are green! This leaves additional testing efforts kind of to the side and doesn't really help with transparency and visibility. I'm not going to mention having an automatically deployed staging environment for every change because very few teams actually have managed to do this effectively.
+1 your wildest ideas in
Issue #700.Speaking of modern engineering flow is your team truly agile? When and how do you plan your testing activities ? Before the devel sprint or afterwards? How many testers take part in refining product backlog and working on user stories?
Similar to GitHub flow lots of teams and open source projects are using Trello to effectively organize their development process. Testing should not be left behind and Kiwi TCMS may be able to help.
devel-test-planning
process for agile teams and what we can do to make this easier for testers.
Please share and +1 your wildest ideas in
Issue #701What makes a test engineer productive when they need to assess product risk and new features, when mapping product requirements documents (PRD) to test plans and test cases, when collaborating on user stories and behavior specification ? What makes developers, product owners, designers and other professionals productive when it comes to dealing with testing ?
For example consider the following workflow:
Later we iterate through the sprints and for each sprint something like this happens:
Devel is also part of testing, right? Product owners, UX and interaction designers as well. Producing quality software product is a team effort!
In every step of the way Kiwi TCMS can provide notification wizards, guidelines and/or documentation for best practices, facilitate tooling, e.g. to write user stories and assess them or map out an exploratory testing session, etc. The list of ideas is virtually endless. We can even go into deep learning, AI and blockchain but honestly who knows how to use them in testing ?
Our team is not quite sure how this goal will look like 3 months from now but we are certain that testing needs to happen first, last and all the time during the entire software development lifecycle. By providing the necessary functionality and tools in Kiwi TCMS we can boost engineering productivity and steer the testing process in your organization into a better, more productive direction which welcomes participation from all engineering groups.
Let's consider another point of view: testing is a creative activity which is benefited by putting your brain into a specific state of mind! For example Gherkin (the Given-When-Then language) has the benefit of forcing you to think about behavior and while doing so you are vocalizing the various roles in the system, what kind of actions are accepted and what sort of result is expected! Many times this will help you remember or discover missing scenarios, edge cases and raise even more questions!
Crazy ideas, brain dumps and +1 as always are welcome in
Issue #703.
Coding alone is not fun! Here's what you can do to help us:
Star button on our GitHub repository+1 reaction on GitHub issues (top-right corner)We are also looking to expand our core team and the list of occasional contributors. The following are mostly organizational goals:
labelOur team will be working on areas related to the goals above. A +1
reaction on GitHub issues will help us prioritize what we work on!
Bug fixes and other issues will be occasionally slipped into the stream and pull requests from non-team contributors will be reviewed and merged in a timely fashion.
There is at least 1 full day of work that goes behind the scenes when a new version is officially released: compile changelog, build images and upload them, create blog post and newsletter announcement, share on social media, etc. We also deploy on our own Kiwi TCMS instance as a stop-gap measure before making everything public!
New PyPI tarballs and Docker images will be released every few weeks as we see fit, this has been our standard process. We try to align releases with Django's release schedule and try to cut a new version when there are known security vulnerabilities fixed. However we can't guarantee this will always be the case!
If you are in a hurry and need something quickly the best option is to send a pull request, build your own Docker image from source and maybe consider sponsoring us via Open Collective!
Happy testing!
We're happy to announce Kiwi TCMS version 6.4! This is a security, improvement and bug-fix update that includes new versions of Django, Patternfly and other dependencies. You can explore everything at https://public.tenant.kiwitcms.org!
Supported upgrade paths:
5.3 (or older) -> 5.3.1 5.3.1 (or newer) -> 6.0.1 6.0.1 -> 6.1 6.1 -> 6.1.1 6.1.1 -> 6.2 (or newer)
Docker images:
kiwitcms/kiwi latest 39fcb88182bb 963.4 MB kiwitcms/kiwi 6.2 7870085ad415 957.6 MB kiwitcms/kiwi 6.1.1 49fa42ddfe4d 955.7 MB kiwitcms/kiwi 6.1 b559123d25b0 970.2 MB kiwitcms/kiwi 6.0.1 87b24d94197d 970.1 MB kiwitcms/kiwi 5.3.1 a420465852be 976.8 MB
If you are using Kiwi TCMS as a Docker container then:
cd Kiwi/ git pull docker-compose down docker pull kiwitcms/kiwi docker pull centos/mariadb docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate
Don't forget to backup before upgrade!
WARNING: kiwitcms/kiwi:latest and docker-compose.yml will always point to the latest available version! If you have to upgrade in steps, e.g. between several intermediate releases, you have to modify the above workflow:
# starting from an older Kiwi TCMS version docker-compose down docker pull kiwitcms/kiwi:<next_upgrade_version> edit docker-compose.yml to use kiwitcms/kiwi:<next_upgrade_version> docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate # repeat until you have reached latest
Happy testing!
Hello everyone, in this article I will outline the progress that the Kiwi TCMS team has made towards achieving the goals in our 2018 roadmap (mid-year update here). TLDR; goals are completed at 62%. Refactoring legacy code is showing good results, less so on the front-end side and there are items still in progress!
Status: good progress
Initially CodeClimate reported a "D" rating with 600+ code smells and 600+ duplications and a 1 year estimation to resolve these. We're now down to "C" rating with 171 smells and 203 duplications.
The level of technical debt has dropped from 32.5% down to 17.7% and we have removed around 14000 lines of Python code and 8000 lines of JavaScript code without losing significant functionality.
Checkout the stats for more info!
Status: almost finished
Both pylint and pylint-django have been integrated into our CI workflow. There are even
some custom built plugins that we use. The number of issues reported is down to 100
from 4000+ initially. These are predominantly fixme comments which are also in parts
of the code that are scheduled for removal and refactoring.
Status: moderate progress
Several views were modified to return pure JSON but we've not done any targeted work to resolve this issue. A number of other views have been removed in favor of using the existing JSON-RPC layer.
This is an internal refactoring effort which isn't very visible from the outside. This is also one of the factors contributing to the high number of removed source code.
Status: no progress
Not much has been done in this area except the occasional refactoring to JSON-RPC.
Status: complete
Status: moderate progress, dropped
All RPC methods have been documented! The rest of the internals will be documented as we go along.
Status: good progress
We still carry around jQuery, jQuery-UI and Handlebars.js. They will be removed once the pages using them are converted to use the Patternfly widgets library.
Status: moderate progress
There are still over 50 HTML templates in tcms/templates/ that need to be
refactored into Patternfly. We've been working on them one at a time and will
focus more on this effort in the next couple of months.
Status: moderate progress
Some of the pages have been converted to use Patternfly. The most important pages that still have a different look and feel are TestPlan view, TestCase view and TestRun view. These are also the hardest to convert because they have lots of tabs/components which pull information from various places. Our goal is to create reusable widgets for the various components (e.g. a list of TestCases) and then include these components into several different templates to minimize code duplication.
Status: moderate progress
A number of JavaScript functions have been refactored and removed during the past few releases but there are still thousands of lines of code left to deal with. This effort is mostly happening in parallel with the Patternfly redesign. We still don't have anything to test front-end JavaScript functionality!
Status: good progress
We are seeing a steady stream of new users registered on https://public.tenant.kiwitcms.org and there are several active contributors on GitHub. Most of our translators are very active and keep their respective languages fresh and up to date!
Kiwi TCMS was represented at OSCAL Tirana, DjangoCon Heidelberg, PyCon Prague, HackConf Sofia, PiterPy St. Petersburg and OpenFest Sofia. We've also been approved for a project stand at FOSDEM 2019 so watch this blog for more news.
Happy testing!
We're happy to announce Kiwi TCMS version 6.3! This is a medium severity security update that includes new versions of Django and Patternfly, new database migrations, lots of improvements, bug fixes and internal refactoring. This version also introduces integration with GitLab issue tracker! You can explore everything at https://public.tenant.kiwitcms.org!
Supported upgrade paths:
5.3 (or older) -> 5.3.1 5.3.1 (or newer) -> 6.0.1 6.0.1 -> 6.1 6.1 -> 6.1.1 6.1.1 -> 6.2 (or newer)
Docker images:
kiwitcms/kiwi latest 8135624fdca2 960.3 MB kiwitcms/kiwi 6.2 7870085ad415 957.6 MB kiwitcms/kiwi 6.1.1 49fa42ddfe4d 955.7 MB kiwitcms/kiwi 6.1 b559123d25b0 970.2 MB kiwitcms/kiwi 6.0.1 87b24d94197d 970.1 MB kiwitcms/kiwi 5.3.1 a420465852be 976.8 MB
These actions have always been a bit broken and didn't check the correct permission labels. You can still update items individually!
If you are using Kiwi TCMS as a Docker container then:
cd Kiwi/ git pull docker-compose down docker pull kiwitcms/kiwi docker pull centos/mariadb docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate
Don't forget to backup before upgrade!
WARNING: kiwitcms/kiwi:latest and docker-compose.yml will always point to the latest available version! If you have to upgrade in steps, e.g. between several intermediate releases, you have to modify the above workflow:
# starting from an older Kiwi TCMS version docker-compose down docker pull kiwitcms/kiwi:<next_upgrade_version> edit docker-compose.yml to use kiwitcms/kiwi:<next_upgrade_version> docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate # repeat until you have reached latest
Happy testing!
We're happy to announce Kiwi TCMS version 6.2.1! This is a small release that includes some improvements and bug-fixes. You can explore everything at https://public.tenant.kiwitcms.org!
Supported upgrade paths:
5.3 (or older) -> 5.3.1 5.3.1 (or newer) -> 6.0.1 6.0.1 -> 6.1 6.1 -> 6.1.1 6.1.1 -> 6.2 (or newer)
Docker images:
kiwitcms/kiwi latest 24338088bf46 956.8 MB kiwitcms/kiwi 6.2 7870085ad415 957.6 MB kiwitcms/kiwi 6.1.1 49fa42ddfe4d 955.7 MB kiwitcms/kiwi 6.1 b559123d25b0 970.2 MB kiwitcms/kiwi 6.0.1 87b24d94197d 970.1 MB kiwitcms/kiwi 5.3.1 a420465852be 976.8 MB
If you are using Kiwi TCMS as a Docker container then:
cd Kiwi/ git pull docker-compose down docker pull kiwitcms/kiwi docker pull centos/mariadb docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate
Don't forget to backup before upgrade!
WARNING: kiwitcms/kiwi:latest and docker-compose.yml will always point to the latest available version! If you have to upgrade in steps, e.g. between several intermediate releases, you have to modify the above workflow:
# starting from an older Kiwi TCMS version docker-compose down docker pull kiwitcms/kiwi:<next_upgrade_version> edit docker-compose.yml to use kiwitcms/kiwi:<next_upgrade_version> docker-compose up -d docker exec -it kiwi_web /Kiwi/manage.py migrate # repeat until you have reached latest
Happy testing!