Kiwi TCMS 6.1

We're happy to announce Kiwi TCMS version 6.1! This release introduces new database migrations, internal updates and bug fixes. It is a small release designed to minimize the number of database migrations by squashing them together. You can explore everything at https://demo.kiwitcms.org.

NOTE: there is the 6.0.1 release which resolves an upgrade problem caused by non-applied migrations which have been later squashed and released in the same release! It is best to jump through the intermediate releases to ensure a smooth upgrade!

Supported upgrade paths:

5.3   (or older) -> 5.3.1
5.3.1 (or newer) -> 6.0.1
6.0.1            -> 6.1

Docker images:

kiwitcms/kiwi       latest  b559123d25b0    970.2 MB
kiwitcms/kiwi       6.1     b559123d25b0    970.2 MB
kiwitcms/kiwi       6.0.1   87b24d94197d    970.1 MB
kiwitcms/kiwi       5.3.1   a420465852be    976.8 MB

Changes since Kiwi TCMS 6.0

Improvements

  • Update Jira from 1.0.10 to 2.0.0
  • Update to Patternfly 3.55.0
  • Use button instead of HTML link for deleting test plan (Oleg Kainov)
  • New middleware that will check missing settings. At the moment will only check Base URL configuration which often gets forgotten!

Bug fixes

  • Hot-fix for error caused by the API method TestRun.update. Error was initially reported on StackOverflow. This patch makes it possible to use the API without crashing however the TestRun.update method doesn't handle the stop_date field at the moment!

Translations

Refactoring

  • Fix pylint errors (Ivaylo Ivanov)
  • Remove unused TestRun.list and TestCase.list_confirmed methods
  • Remove unused plan_by_id_or_name() and is_int(). Fixes Issue #269

Database

  • Rename tcms.core.contrib.auth to tcms.kiwi_auth
  • Remove field user from TestCaseTag, TestRunTag and TestPlanTag models
  • Squash migrations for management app
  • Squash migrations for testcases app
  • Squash migrations for testplans app
  • Squash migrations for testruns app

How to upgrade

If you are using Kiwi TCMS as a Docker container then:

cd Kiwi/
git pull
docker-compose down
docker pull kiwitcms/kiwi
docker pull centos/mariadb
docker-compose up -d
docker exec -it kiwi_web /Kiwi/manage.py migrate

Don't forget to backup before upgrade!

Subscription based online service

We are planning to convert https://demo.kiwitcms.org into a subscription based service for people who don't want to bother installing Kiwi TCMS on their own systems and just want to use it for test collaboration with their team.

The income from this service will go towards covering operating costs and supporting the development team. Watch this blog for further announcements about this. Meanwhile you may share your thoughts via Issue #515.

Plans to remove deprecated functionality

TestRun.product_version will be removed from the database schema in favor of TestRun.plan.product_version! With the latest web interface it is not possible to select a product version different than the one selected in the Test Plan. Test runs should only be able to select product builds (after Issue #246 is fixed).

Kiwi TCMS team updates

I am happy to announce that our team is steadily growing! As we work through our roadmap, status update here, and on-board new team members I start to feel the need for a bit more structure and organization behind the scenes. I also wish for consistent contributions to the project (commit early, commit often) so I can better estimate the resources that we have!

I am also actively discussing Kiwi TCMS with lots of people at various conferences and generate many ideas for the future. The latest SEETEST in Belgrade was particularly fruitful. Some of these ideas are pulling into different directions and I need help to keep them under control!

Development-wise sometimes I lose track of what's going on and who's doing what between working on Kiwi TCMS, preparing for conferences and venues to promote the project, doing code review of other team members, trying not to forget to check-in on progress (especially by interns), recruiting fresh blood and thinking about the overall future of the project. Our user base is growing and there are days where I feel like everything is happening at once or that something needs to be implemented ASAP (which is usually true anyway)!

Meet Rayna Stankova in the role of our team coach! Reny is a director for Women Who Code Sofia, senior QA engineer at VMware, mentor with CoderDojo Bulgaria and a long-time friend of mine. Although she is an experienced QA in her own right she will be contributing to the people side of Kiwi TCMS and less so technically!

Her working areas will be planning and organization:

and generally serving as another very experienced member of the team!

We did a quick brainstorming yesterday and started to produce results (#smileyface)! We do have a team docs space to share information (non-public for now, will open it gradually as we grow) and came up with the idea to use Kiwi TCMS as a check-list for our on-boarding/internship process!

I don't know how it will play out but I do expect from the team to self-improve, be inspired, become more focused and more productive! All of this also applies to myself, even more so!

Existing team members progress

Last year we started with 2 existing team members (Tony and myself) and 3 new interns (Ivo, Kaloyan and Tseko) who built this website!

Tony is the #4 contributor to Kiwi TCMS in terms of number of commits and is on track to surpass one of the original authors (before Kiwi TCMS was forked)! He's been working mostly on internal refactoring and resolving the thousands of pylint errors that we had (down to around 500 I think). This summer Tony and I visited the OSCAL conference in Tirana and hosted an info booth for the project.

Ivo is the #5 contributor in terms of numbers of commits. He did learn very quickly and is working on getting rid of the remaining pylint errors. His ability to adapt and learn is quite impressive actually. Last month he co-hosted a git workshop at HackConf, a 1000+ people IT event in Sofia.

Kaloyan did most of the work on our website initially (IIRC). Now he is studying in the Netherlands and not active on the project. We are working to reboot his on-boarding and I'm hoping he will find the time to contribute to Kiwi TCMS regularly.

From the starting team only Tseko decided to move on to other ventures after he contributed to the website.

Internship program

At Kiwi TCMS we have a set of training programs that teach all the necessary technical skills before we let anyone actively work on the project, let alone become a team member.

Our new interns are Denitsa Uzunova and Desislava Koleva. Both of them are coming from Vratsa Software Community and were mentors at the recently held CodeWeek hackathon in their home city! I wish them fast learning and good luck!

Happy testing!

Kiwi TCMS winter conference presence

We are happy to announce that OpenFest - the biggest open source conference in Bulgaria has provided an info booth for our project. This year the event will be held on 3rd and 4th of November at Sofia Tech Park!

Last time the team went to a conference together we had a lot of fun! Join us at OpenFest to learn more about Kiwi TCMS and have fun with us!

In case you are unable to visit Sofia, which you totally should, you can catch up with us in Russia until the end of the year:

Feel free to ping us at @KiwiTCMS or look for the kiwi bird logo and come to say hi. Happy testing!

Kiwi TCMS 6.0

We're happy to announce Kiwi TCMS version 6.0! This release introduces new database migrations, removal of environment properties in favor of tags, internal updates and bug fixes. You can explore everything at https://demo.kiwitcms.org.

This release removes squashed migrations. Direct upgrades from versions < 5.3.1 to 6.0 will break without an intermediate upgrade to 5.3.1!

Docker images:

kiwitcms/kiwi       latest  cd087297f349    991.3 MB

Changes since Kiwi TCMS 5.3.1

Improvements

  • Update to Django 2.1.2 due to high severity security issue
  • Update to Patternfly 3.54.8
  • Tag objects are now shown in Admin panel
  • Add autocomplete when adding tags to TestRun via UI

Removed functionality

  • TestCase new and edit views no longer allow editing of tags. Tags can be added/removed from the Tags tab which also makes sure to properly account for permissions
  • Remove EnvGroup, EnvProperty and EnvValue models in favor of tags. Existing values and properties are converted into tags and automatically added to test runs!
  • Convert squashed database migrations to regular ones and remove older migrations. WARNING: upgrade from versions <= 5.3.1 to 6.0 will break without an intermediate upgrade to kiwitcms/kiwi:5.3.1 a420465852be.
  • Remove deprecated TestCase.estimated_time and TestRun.estimated_time. Fixes Issue #514

Backend API

  • No longer use product_version for TestRun.create. Fixes Issue #522
    • 'product' is no longer required
    • 'product_version' is no longer required
    • 'manager' and 'default_tester' can be usernames or IDs
  • TestCase.create no longer accepts 'tag' values
  • TestRun.add_tag and TestRun.remove_tag now return list of tags. Previously these methods returned None! This is the list of tags assigned to the TestRun that is being modified!

Bug fixes

  • Fix mismatched HTML tag in plan/get.html template (Oleg Kainov)
  • Don't use |slugify filter in templates which breaks HTML links with non-ASCII TestPlan names. Fixes Sentry KIWI-TCMS-38

Refactoring

  • Fix pylint errors (Ivaylo Ivanov, Anton Sankov)
  • Use existing JSON-RPC methods to add/remove tags via webUI and remove specialized backend methods that handled these requests. Also make sure to obey respective permissions

Translations

How to upgrade

If you are using Kiwi TCMS as a Docker container then:

cd Kiwi/
git pull
docker-compose down
docker pull kiwitcms/kiwi
docker pull centos/mariadb
docker-compose up -d
docker exec -it kiwi_web /Kiwi/manage.py migrate

Don't forget to backup before upgrade!

Subscription based online service

We are planning to convert https://demo.kiwitcms.org into a subscription based service for people who don't want to bother installing Kiwi TCMS on their own systems and just want to use it for test collaboration with their team.

The income from this service will go towards covering operating costs and supporting the development team. Watch this blog for further announcements about this. Meanwhile you may share your thoughts via Issue #515.

Plans to remove deprecated functionality

TestRun.product_version will be removed from the database schema in favor of TestRun.plan.product_version! With the latest web interface it is not possible to select a product version different than the one selected in the Test Plan. Test runs should only be able to select product builds (after Issue #246 is fixed).

Happy birthday Kiwi TCMS

1 year infographic

How to add API logging for Kiwi TCMS

In this blog post I will show more ways to customize Kiwi TCMS by adding logging capabilities to the API backend. Indeed this is a feature that our team deemed not required for upstream and was removed in PR #436.

Start by creating the following directory structure:

    api_logging/
        __init__.py
        handlers.py
        models.py

This is a small Django application that will log every call to the API backend. Each file looks like this:

    # models.py contains DB schema for your new table
    from django.db import models
    from django.conf import settings

    class ApiCallLog(models.Model):
        executed_at = models.DateTimeField(auto_now_add=True)
        user = models.ForeignKey(settings.AUTH_USER_MODEL, null=True, blank=True,
                                 on_delete=models.CASCADE)
        method = models.CharField(max_length=255)
        args = models.TextField(blank=True)

        def __str__(self):
            return "%s: %s" % (self.user, self.method)

Then

    # handlers.py overrides the RPC handlers coming from django-modernrpc
    from modernrpc import handlers

    from django.conf import settings
    from django.contrib.auth import get_user_model

    from .models import ApiCallLog

    def log_call(request, method_name, args):
        """ Log an RPC call to the database or stdout in DEBUG mode. """
        request_user = request.user
        if not request_user.is_authenticated:
            # create an anonymous user object for logging purposes
            request_user, _ = get_user_model().objects.get_or_create(
                username='Anonymous',
                is_active=False)

        if method_name is None:
            method_name = '--- method_name missing ---'

        if settings.DEBUG:
            print('API call:: user: {0}, method: {1}, args: {2}'.format(
                request_user,
                method_name,
                args))
        else:
            ApiCallLog.objects.create(
                user=request_user,
                method=method_name,
                args=str(args))

    class XMLRPCHandler(handlers.XMLRPCHandler):
        def process_request(self):
            encoding = self.request.encoding or 'utf-8'
            data = self.request.body.decode(encoding)
            params, method_name = self.loads(data)

            log_call(self.request, method_name, params)
            return super().process_request()

    class JSONRPCHandler(handlers.JSONRPCHandler):
        def process_single_request(self, payload):
            method_name = payload.get('method', None)
            params = payload.get('params')

            log_call(self.request, method_name, params)
            return super().process_single_request(payload)

NOTE: You will have to execute ./manage.py makemigrations api_logging to create the initial migration for Django. This could be easier if you place the above directory into existing Django application or craft the migration file by hand!

The last thing you want to do is create a local_settings.py file which will override Kiwi TCMS defaults:

    # local_settings.py
    from django.conf import settings

    settings.INSTALLED_APPS += [
        'api_logging',
    ]

    MODERNRPC_HANDLERS = ['api_logging.handlers.XMLRPCHandler',
                          'api_logging.handlers.JSONRPCHandler']

Then place everything in Dockerfile like so:

    FROM kiwitcms/kiwi

    COPY ./api_logging/ /venv/lib64/python3.6/site-packages/api_logging/
    COPY local_settings.py /venv/lib64/python3.6/site-packages/tcms/settings/

Kiwi TCMS will import your local_settings.py and enable the logging application. Now build your customized Docker image and use it for deployment!

Happy testing!

Kiwi TCMS 5.3.1

We're happy to announce Kiwi TCMS version 5.3.1! This release brings lots of UI updates and removal of unused and/or duplicated functionality and source code. Many pages have been redesigned with the Patternfly library to have a modern look and feel which you can experience at https://demo.kiwitcms.org.

This will be the last release to carry around squashed migrations. In version 6.0 older migrations will be deleted and replaced with their squashed equivalents. Upgrades from versions < 5.3 to 6.0 will break without an intermediate upgrade to 5.3.1!

Use kiwitcms/kiwi:5.3.1 from Docker Hub when upgrading at some point in the future!

Docker images:

kiwitcms/kiwi        5.3.1               a420465852be        976.8 MB
kiwitcms/kiwi        latest              a420465852be        976.8 MB

Changes since Kiwi TCMS 5.2

Improvements

  • Update to Django 2.1.1
  • Update Patternfly version. Fixes Issue #381
  • Replace TinyMCE with SimpleMDE markdown editor. Used for TestPlan text and TestCase action, setup, breakdown, effect fields
  • Allow downstream builds to customize the login templates by providing registration/custom_login.html template. It can either override the entire login page or provide additional information inside the custom_login block!

Known issues

  • You may need to strip existing texts from HTML tags that were generated by TinyMCE. Either do this manually as you go along and use/edit records or in bulk via the API and tools like html2text. We strongly advise you to backup your database before attempting to automatically edit text fields!

Visual changes

  • Remove breadcrumbs at the top of pages. Only admin pages still have them
  • Convert login and registration templates to Patternfly. Fixes Issue #211
  • Convert 404 and 500 templates to Patternfly
  • Convert dashboard page to Patternfly
  • Convert TestRun new, edit and clone pages to Patternfly. Fixes Issue #17
  • Convert Search Test Plans page to Patternfly
  • Convert Search Test Runs page to Patternfly
  • Convert Search Test Cases page to Patternfly
  • TestPlan view page, Runs tab now re-uses the search form for test runs which is built using Patternfly
  • Added pagination controls to pages with search results

Removed functionality

  • When creating or editing TestRun
    • field estimated_time is scheduled for removal and is not shown
    • product_version is scheduled for removal in favor of TR.plan.product_version
    • Product & Version can't be edited anymore. Must be set on the parent TestPlan instead. Still allows to specify builds
  • Remove the ability to clone multiple TestPlans from search results
  • Remove the ability to upload TestPlan document files in favor of the existing API
  • Remove TestCase export to XML in favor of existing API
  • Remove Advanced Search functionality. Fixes Issue #448, Issue #108
  • Remove tech preview feature: Django Report Builder

Translations

Bug fixes

  • Don't use get_full_url() where not needed. Closes Issue #380
  • Remove unused templates. Fixes Issue #114
  • Submit filter form when clicking on tag value. Fixes Issue #426
  • Update TestCaseRun.tested_by when setting status. Fixes Issue #459
  • Add tests for KiwiUserAdmin. Closes Issue #489

Settings

  • The following settings have been removed MOTD_LOGIN, WELCOME_MESSAGE and TINYMCE_DEFAULT_CONFIG

Refactoring

  • Fix pylint errors (Anton Sankov, Ivaylo Ivanov)
  • Remove lots of unused functions and classes
  • Remove old or unnecessary templates
  • Remove html2text dependency
  • Remove unused CSS and vendored-in JavaScript libraries
  • Add JavaScript JSON-RPC client which is now used by the front-end to communicate with the existing JSON-RPC API on the back-end. This replaces many 'ajax' views which are only used to render the UI and were duplicating functionality with existing API
  • Non dist/ files are no longer removed from node_modules/ when building a docker image because packages like moment.js and bootstrap-datetimepicker.js don't ship their files in dist/
  • Convert TestPlans.TreeView to JSON RPC

How to upgrade

If you are using Kiwi TCMS as a Docker container then:

cd Kiwi/
git pull
docker-compose down
docker pull kiwitcms/kiwi
docker pull centos/mariadb
docker-compose up -d
docker exec -it kiwi_web /Kiwi/manage.py migrate

Don't forget to backup before upgrade!

Ask your questions on StackOverflow

Kiwi TCMS is now using the 'kiwi-tcms' tag on StackOverflow to track questions. Please submit your support requests and general questions to StackOverflow tagged with the 'kiwi-tcms' tag! Support requests via GitHub or email will not be answered anymore (GitHub issues are for bugs or feature discussions only)!

Subscription based online service

We are planning to convert https://demo.kiwitcms.org into a subscription based service for people who don't want to bother installing Kiwi TCMS on their own systems and just want to use it for test collaboration with their team.

The income from this service will go towards covering operating costs and supporting the development team. Watch this blog for further announcements about this. Meanwhile you may share your thoughts via Issue #515.

Plans to remove deprecated functionality

TestRun.product_version will be removed from the database schema in favor of TestRun.plan.product_version! With the latest web interface it is not possible to select a product version different than the one selected in the Test Plan. Test runs should only be able to select product builds (after Issue #246 is fixed).

TestRun.estimated_time is scheduled for removal if not used. Objections and use-cases must be shared via Issue #514.

EnvGroup and EnvProperty models are scheduled for removal in favor of Tag. Internally tags are referenced in more places inside Kiwi TCMS' source code and they appear to be more fully developed. On the other hand while environment appears very similar to tags it is not exactly the same. Ongoing discussion is happening via Issue #484.

Kiwi TCMS 5.2

We're happy to announce Kiwi TCMS version 5.2! This release introduces new database migrations and converts the Docker image to a non-root user with uid 1001. You may have to adjust ownership/permissions on the kiwi_uploads Docker volume!

The new kiwitcms/kiwi:latest docker image has Image ID 6bc4e3124fd4. https://demo.kiwitcms.org has also been updated!

Changes since Kiwi TCMS 5.1

Enhancements

  • Upgrade to Django 2.1
  • Upgrade to django-report-builder 6.2.2, compatible with Django 2.1
  • Docker image now executes with uid 1001 instead of root
    • image based on centos7 image instead of centos/httpd
    • image now exposes ports 8080 and 8443
    • Apache logs now printed on Docker console
    • SSL certificates copied to /Kiwi/ssl inside Docker image instead of being bind-mounted
    • uploads dir changed to /Kiwi/uploads
    • static dir changed to /Kiwi/static
    • /Kiwi is now owned by uid 1001
    • /venv is now owned by uid 1001
    • docker-compose.yml is updated to match
  • Fix pylint errors (Ivaylo Ivanov)
  • Allow users to see other profiles via Admin
  • Use password change form from Admin instead of custom one
  • product.py will try to import local_settings.py if available in the same directory. This can be used to customize settings in downstream distributions
  • Updated Slovenian translation

Bug fixes

  • Make password reset views public
  • Don't crash when adding new users via Admin

Refactoring

  • Remove UserProfile model. Kiwi TCMS doesn't needs extra information about users so we remove this part of the application. Custom installations may choose to define their own profiles if they wish
  • Remove custom DBModelBackend authentication backend
  • Remove unused tcms.core.context_processors.auth_backend_processor
  • Remove unused get_using_backend(). Fixes Issue #261
  • Remove dj_pagination. Fixes Issue #110

Settings

  • AUTHENTICATION_BACKENDS is removed
  • PAGINATION_DEFAULT_PAGINATION is removed
  • Navigation menu links are now defined in MENU_ITEMS and can be redefined

Signals

  • USER_REGISTERED_SIGNAL now doesn't receive the backend parameter

How to upgrade

If you are using Kiwi TCMS as a Docker container then:

cd Kiwi/
git pull
docker-compose down
docker pull kiwitcms/kiwi
docker pull centos/mariadb
docker-compose up -d
docker exec -it kiwi_web /Kiwi/manage.py migrate

Don't forget to backup before upgrade!

How to override templates for Kiwi TCMS

This is the first publication in our customization series. It will show you how to override any template used by Kiwi TCMS. As an example we will override the email template that is used when registering new account. By default the email text looks like this:

    Welcome {{ user }},
    thank you for signing up for an {{ site_domain }} account!

    To activate your account, click this link:
    {{ confirm_url }}

https://demo.kiwitcms.org runs a custom Docker image based on kiwitcms/kiwi. For this image the confirmation email looks like this

    Welcome {{ user }},
    thank you for signing up for our Kiwi TCMS demo!

    To activate your account, click this link:
    {{ confirm_url }}

    GDPR no longer allows us to automatically subscribe you to
    our newsletter. If you wish to keep in touch and receive emails
    with news and updates around Kiwi TCMS please subscribe at:
    https://kiwitcms.us17.list-manage.com/subscribe/post?u=9b57a21155a3b7c655ae8f922&id=c970a37581

    --
    Happy testing!
    The Kiwi TCMS team
    http://kiwitcms.org

The file that we want to override is tcms/templates/email/confirm_registration.txt.

Create a local directory (git repository) which will hold customization configuration and create a file named templates.d/email/confirm_registration.txt with your text!

Next you want to make this file available inside your docker image so your Dockerfile should look like this:

    FROM kiwitcms/kiwi

    COPY ./templates.d/ /venv/lib64/python3.6/site-packages/tcms/overridden_templates/
    COPY local_settings.py /venv/lib64/python3.6/site-packages/tcms/settings/

where local_settings.py contains

    import os
    from django.conf import settings

    settings.TEMPLATES[0]['DIRS'].insert(0, os.path.join(settings.TCMS_ROOT_PATH, 'overridden_templates'))

The following code states instruct Django to look into overridden_templates first and use any templates it finds there; also make my files available in that specific location inside the docker image.

This approach can be used for all templates that you wish to override. Take into account that file names must match (Django searches templates by directory path). Now build your customized Docker image and use that for deployment!

Happy testing!

Kiwi TCMS 5.1

We're happy to announce Kiwi TCMS version 5.1! This release introduces new database migrations, an experimental reporting feature, deprecated functionality and bug fixes.

The new kiwitcms/kiwi:latest docker image has Image ID cc9b55aa664d. https://demo.kiwitcms.org has also been updated!

Changes since Kiwi TCMS 5.0

Enhancements

  • Integrate with Django Report Builder as tech-preview. This makes it possible for power users and administrators to generate tabular reports. You will have to know the existing DB schema if you want to create your own reports. See http://kiwitcms.readthedocs.io/en/latest/db.html. This feature is in tech-preview and it may be removed if it doesn't work out. Please comment at: Issue #452.
  • Allow using manage.py dumpdata|loaddata|sqlflush|dbshell for backups, see blog post
  • In TestCase view page add a link to delete the current test case
  • In TestCase Admin page the + Add TestCase button now allows to create new test case
  • The version menu item in the helper menu now links to Changelog

Deprecated functionality

  • Start showing deprecation warning for Advanced search, see Issue #448

Bug fixes

  • Allows Product/Version/Build to be shown in Testing Report. Fixes Sentry KIWI-TCMS-2C
  • Default to https:// links if not running locally. Fixes Issue #450
  • Apply missing CSS class for object history table so it can be displayed nicely

Refactoring

  • Squash some database migrations
  • Fix a number of pylint issues
  • Remove unused testruns.views::view_caseruns(). References Issue #316
  • Remove unused template report/caserun.html
  • Handle TestRun deletion via admin not home grown code

How to upgrade

If you are using Kiwi TCMS as a Docker container then:

cd Kiwi/
git pull
docker-compose down
docker pull kiwitcms/kiwi
docker pull centos/mariadb
docker-compose up -d
docker exec -it kiwi_web /Kiwi/manage.py migrate

Don't forget to backup before upgrade!

Help us make Kiwi TCMS better

According to our 2018 roadmap update the Kiwi TCMS team is investing a great deal of our time to make the code base more maintainable! For the past few releases we've been balancing code refactoring with bug-fixing and enhancements and we have lots of ideas down the line.

We want to hear your voice! Tell us how you use Kiwi TCMS inside your organization. We want to know what works for you and what doesn't so we can make it better. Feel free to comment below, file GitHub issues or send us an email.

Happy testing!

How to backup Docker volumes for Kiwi TCMS

When you start Kiwi TCMS by running docker-compose up (see here) it will automatically create 2 volumes: kiwi_db_data and kiwi_uploads. This blog post will outline how to backup these docker volumes.

Backing up the database

Kiwi TCMS is a Django application and the manage.py command provides an easy way to dump and load the database contents. To export all contents on your docker host execute:

docker exec -it kiwi_web /Kiwi/manage.py dumpdata --all --indent 2 > database.json

This will create a file named database.json in the current directory, outside of the running container!

You can restore the database contents by using the following commands:

# delete data from all tables
docker exec -it kiwi_web /bin/bash -c '/Kiwi/manage.py sqlflush | /Kiwi/manage.py dbshell'
# then reload the existing data
cat database.json | docker exec -i kiwi_web /Kiwi/manage.py loaddata --format json -

NOTE: depending on your scenario you may want to remove the existing volume (docker-compose down && docker volume rm kiwi_db_data) and re-create the DB schema (/Kiwi/manage.py migrate) before restoring the contents!

WARNING: the above steps are applicable to Kiwi TCMS 5.1 or above. On earlier versions manage.py will fail due to various issues.

Backing up file uploads

Uploaded files can easily be backed up with:

docker exec -it kiwi_web /bin/tar -cP /var/kiwi/uploads > uploads.tar

and then restored:

cat uploads.tar | docker exec -i kiwi_web /bin/tar -x

You may also try the rsync command but be aware that it is not installed by default!

The same approach may be used to backup /var/lib/mysql/ from the kiwi_db container.

Alternatives

By default both docker volumes created for Kiwi TCMS use the local driver and are available under /var/lib/docker/volumes/<volume_name> on the host running your containers. You can try backing them up from there as well.

Another alternative is to use the docker-lvm-plugin and create these volumes as LVM2 block devices. Then use lvcreate -s command to create a snapshot volume. For more information see chapter 2.3.5. Snapshot Volumes from the LVM Administrator Guide for Red Hat Enterprise Linux 7.

Happy testing!

Mid-year roadmap status report

Hello everyone, in this article I will outline the progress that the Kiwi TCMS team has made towards achieving the goals on our roadmap.

Make code easier to maintain

Status: moderate progress

Initially CodeClimate reported a "D" rating with a 1 year estimated effort. Now it is still on "D" rating with a 7 months estimated effort to bring the project back in shape. Code smells have dropped from 600+ to 418, duplications have been reduced from 600+ to 359! At the same time technical debt ratio has been decreased from 32,5% to 21,6% and little over 10000 lines of code have been removed from the source code. Checkout the stats for more info!

Use pylint and pylint-django

Status: good progress

Both pylint and pylint-django have been integrated into our CI workflow. There are even a few custom built plugins that we use. The number of issues reported is down to around 900 from 4000+ initially. The cleanup has been lead by Anton Sankov with help from Ivaylo Ivanov and myself.

Render HTML, return JSON

Status: no progress

Several views were probably modified to return pure JSON in the meantime but we've not done any targeted work to resolve this issue.

Submit forms, post JSON, GET clean URLs

Status: no progress

Same as above, not much has been done in this area.

API layer

Status: complete

After Kiwi TCMS v4.0 the server side API has been reorganized and updated to follow the model/method names used internally.

After the recent version 5.0 the client side API library has been stripped to its most basic form so that you can work directly with the responses from the server.

There is no more duplication and ambiguity in names because there isn't a lot of code left!

Documentation

Status: moderate progress, dropped

All RPC methods have been documented! The rest of the internals will be documented as we go along.

No vendored JavaScript libraries

Status: moderate progress

Several JavaScript libraries have been removed but we still carry around jQuery and Handlebars.js. No work has been done to convert Kiwi TCMS to use the jQuery version provided with Django.

Less HTML templates with better organization

Status: minimal progress

There are still over 100 HTML templates in Kiwi TCMS. Some of the HTML templates have been merged together, some email templates have been refactored and marked as translatable but the majority of them have not been updated for a long time.

Modern interface with Patternfly

Status: no progress

JavaScript updates and front-end testing

Status: small progress

A number of JavaScript functions have been refactored and removed during the past few releases but there are still thousands of lines of code left to deal with.

Community efforts

Status: moderate progress

We are seeing a steady stream of new users registered on https://demo.kiwitcms.org and there are several active contributors (issues, translations).

Kiwi TCMS was represented at OSCAL Tirana, DjangoCon Heidelberg and PyCon Prague! We're planning to attend HackConf and OpenFest in Sofia by the end of the year.

Happy testing!

Kiwi TCMS 5.0

We're happy to announce Kiwi TCMS and tcms-api version 5.0! This release introduces object history tracking, removal of old functionality and unused code, lots of internal updates and bug fixes.

The new kiwitcms/kiwi:latest docker image has Image ID 468de0abe8a8. https://demo.kiwitcms.org has also been updated!

Changes since Kiwi TCMS 4.2

Settings

  • Remove CACHE because not used
  • Remove PLAN_EMAIL_TEMPLATE, CASE_EMAIL_TEMPLATE and CASE_DELETE_EMAIL_TEMPLATE. Templates can still be overridden if desired

Enhancements

  • Upgrade to Django 2.0.7
  • Allow to delete TestPlan. Fixes Issue #273
  • Don't include username in dashboard URL
  • Copy latest TestPlan text when cloning
  • Always require users to be logged in. Anonymous users will not be allowed access by default. Read-only access to some views (e.g. get TestPlan or TestRun) can be enabled by disabling GlobalLoginRequiredMiddleware! Fixes Issue #230
  • Start tracking change history for TestPlan, TestCase, TestRun and TestCaseRun. Fixes Issue #294
  • History changes are recorded as unified diff which is a universally recognized format
  • Show the actual changes in email notifications. Fixes Issue #199

Bug fixes

  • Fix UnboundLocalError local variable 'message' referenced before assignment. Fixes Sentry KIWI-TCMS-1S
  • Make email address unique when adding users via admin panel. Fixes Issue #352 and Issue #68
  • Fix unsupported operand type(s) for +=: 'int' and 'datetime.timedelta' by initializing timedelta variable properly. Fixes Sentry KIWI-TCMS-1Y
  • Remove core.models.fields with MySQL time conversions. Fixes Issue #390
  • Fix bad JavaScript comparison. Fixes Coverity #289956
  • Remove expression with no effect. Fixes Coverity #289974
  • Rewrite request_host_link() to fix Coverity #289987
  • Fix Coverity #289923 - Typo in identifier
  • Don't send emails for changes performed by myself. Fixes Issue #216

Refactoring

  • Fix pylint issues in several modules (Anton Sankov & Ivaylo Ivanov)
  • Fix wrong Plan Type template variable in advanced search form
  • Do not use Model.objects.update() because it doesn't respect history
  • Use the standard ModelChoiceField instead of custom one
  • Use updateRunStatus() instead of deprecated updateObject()
  • Simplify JavaScript function getInfo() and use it multiple times
  • Simplify previewPlan() by removing unused parameters
  • Unify addChildPlan() and removeChildPlan()
  • Unify getInfoAndUpdateObject() with changeCaseRunAssignee()
  • Unify onTestCaseStatusChange() with changeTestCaseStatus()
  • Convert TestCaseEmailSettings.cc_list to string field
  • Merge report/caseruns_table.html with reports/caseruns.html
  • Rename model UserActivateKey to UserActivationKey. Fixes Issue #276
  • Remove cached_entities(). Fixes Issue #307
  • Remove TestPlanText.checksum field
  • Remove checksum fields for TestCaseText model
  • Remove unused and home-grown template tags
  • Remove unused fields auto_blinddown, description, sortkey from TestCaseRunStatus model. Fixes Issue #186
  • Remove Meta.db_name effectively renaming all tables. New names will use Django's default naming scheme
  • Remove RawSQL queries. We are now 100% ORM based. Fixes Issue #36
  • Remove duplicate MultipleEmailField definition
  • Remove TCMSLog view, TCMSLogManager, TCMSLogModel
  • Remove TestPlanText model, use TestPlan.text instead
  • Remove unused JavaScript files - lib/detetmine_type.js - lib/hole.js - lib/scriptaculous-controls.js.patch - lib/validations.js - static/js/index.js
  • Remove constructPlanParentPreviewDialog()
  • Remove changeCasePriority()
  • Remove changeCaseRunOrder()
  • Remove debug_output() from JavaScript files
  • Remove deprecated /ajax/update/ end-point
  • Remove taggleSortCaseRun()
  • Remove strip_parameters()
  • Remove _InfoObjects.users()
  • Remove get_value_by_type()
  • Remove testcases.views.get_log()
  • Remove mail_scene() methods and related templates

Removed functionality

  • TestRun completion status is no longer updated automatically. You can still update the status manually via the 'Set Finished' or 'Set Running' links! Fixes Issue #367
  • Remove bookmarks functionality. There are many great bookmark manager apps and if the user is keen on bookmarks they should use one of them. Closes Issue #67 and Issue #210
  • Don't track & display history of changes for EnvGroup model
  • Remove Disable/Enable buttons from TestPlan page. Enabling and disabling can still be done via the edit page
  • Remove changeParentPlan() and the ability to change TestPlan parents from the 'Tree View' tab. This can be done via the edit page
  • When viewing a TestPlan the user is no longer able to specify a sorkey for a particular TestCase. Instead they can use the Re-order cases button and move around the entire row of cases to adjust the sort order
  • When working with test case results, inside a TestRun you will not be allowed to change the order of execution. Order should be defined inside the TestPlan instead
  • Remove XmlRpcLog() model. Kiwi TCMS will no longer log RPC calls to the database. This leads to a small performance boost and can be overridden on individual basis if you need to do so.

Translations

  • More source strings marked as translatable
  • New translations for Chinese Simplified, Chinese Traditional, German and Slovenian
  • Stop keeping compiled translations under git. Fixes Issue #387

tcms-api changes since 4.2

  • Requires Python 3.6 or newer because it fixes bugs related to Django's disabling of keep-alive connections. See https://bugs.python.org/issue26402
  • The rpc client is now accessed via TCMS().exec.<Server-Method>
  • Leave only XML-RPC transport classes! This removes the top-level interface behind the API client and the consuming side is left to work with Python dictionaries instead of objects.
  • Remove the interactive tcms script
  • Remove tcms_api.config module
  • Remove logging class
  • Remove script_examples/ directory. These were never tested and maintained

How to upgrade

If you are using Kiwi TCMS as a Docker container then:

cd Kiwi/
git pull
docker-compose down
docker pull kiwitcms/kiwi
docker pull centos/mariadb
docker-compose up -d
docker exec -it kiwi_web /Kiwi/manage.py migrate
docker exec -it kiwi_web /Kiwi/manage.py populate_history --auto

Help us make Kiwi TCMS better

According to our 2018 roadmap the Kiwi TCMS team is investing a great deal of our time to make the code base more maintainable! For the past few releases we've been balancing code refactoring with bug-fixing and minor enhancements and we have lots of ideas down the line.

We want to hear your voice! Tell us how you use Kiwi TCMS inside your organization. We want to know what works for you and what doesn't so we can make it better. Feel free to comment below, file GitHub issues or send us an email.

Happy testing!

Kiwi TCMS 4.2

We're happy to announce Kiwi TCMS and tcms-api version 4.2! This is a security, bug-fix and enhancement update which upgrades to the latest Django version under Python 3.6. We've pushed new kiwitcms/kiwi:latest docker image to Docker Hub and updated the demo instance at https://demo.kiwitcms.org!

This version also includes GDPR related changes which affect our project. Read below for the details.

Changes since Kiwi TCMS 4.1.4

IMPORTANT: this release introduces new database migrations

Security

Settings

Enhancements

Bug fixes

Translations

Documentation

Models and database migrations

GDPR related

In addition the following changes were deployed to our infrastructure:

We're trying to be respectful to your private data and not expose that to any 3rd parties or use it for means other than making Kiwi TCMS better open source software. If you spot areas for improvement please report an issue so we can take a look at it!

tcms-api changes since 4.0.0

How to upgrade

If you are using Kiwi TCMS as a Docker container then

cd Kiwi/
git pull
docker-compose down
docker pull kiwitcms/kiwi
docker pull centos/mariadb
docker-compose up -d
docker exec -it kiwi_web /Kiwi/manage.py migrate

Help us make Kiwi TCMS better

According to our 2018 roadmap the Kiwi TCMS team is investing a great deal of our time to make the code base more maintainable! For the past few releases we've been balancing code refactoring with bug-fixing and minor enhancements and we have lots of ideas down the line.

We want to hear your voice! Tell us how you use Kiwi TCMS inside your organization. We want to know what works for you and what doesn't so we can make it better. Feel free to comment below, file GitHub issues or send us an email.

Happy testing!

Kiwi TCMS conference presence

Kiwi TCMS is going on a small conference tour. This is where you can find us in the next couple of months:

For all of the 3 conferences we're going to have a project presence. In addition to that you can catch-up with Alex Todorov, Kiwi's project lead at: TestCon Moscow(17-19 April, Moscow), Romanian Testing Conference(9-11 May, Cluj-Napoca), PyCon CZ(1-3 June, Prague) and DEVit(10-11 June, Thessaloniki).

If you can ping us at @KiwiTCMS or look for the kiwi bird logo and come to say hi!

Kiwi TCMS 4.1.4

We're happy to announce Kiwi TCMS version 4.1.4! This is a bug-fix and enhancement update which upgrades to the latest Django version, resolves several issues and includes lots of internal code updates.

We've pushed new kiwitcms/kiwi docker image to Docker Hub and updated the demo instance at https://demo.kiwitcms.org!

Changes since Kiwi TCMS 4.1.3

Enhancements

Signals

BREAKING CHANGES:

Bug fixes

Refactoring

How to upgrade

If you are using Kiwi TCMS as a Docker container then

cd Kiwi/
git pull
docker-compose down
docker pull kiwitcms/kiwi
docker pull centos/mariadb
docker-compose up -d
docker exec -it kiwi_web /Kiwi/manage.py migrate

Help us make Kiwi TCMS better

According to our 2018 roadmap the Kiwi TCMS team is investing a great deal of our time to make the code base more maintainable! For the past few releases we've been balancing code refactoring with bug-fixing and minor enhancements and we have lots of ideas down the line.

We want to hear your voice! Tell us how you use Kiwi TCMS inside your organization. We want to know what works for you and what doesn't so we can make it better. Feel free to comment below, file GitHub issues or send us an email.

Happy testing!

Kiwi TCMS 4.1.3

We're happy to announce Kiwi TCMS version 4.1.3! This is a bug-fix and enhancement update which upgrades to the latest Django version, provides easier way for Docker administrators to override SSL certificates and access httpd log files and resolves several issues. This release also includes lots of internal code refactoring which removes code duplication and brings down CodeClimate metrics to 26.9% of technical debt or 10 months to fix.

We've pushed the new kiwitcms/kiwi docker image to Docker Hub and updated the demo instance at https://demo.kiwitcms.org. Starting forward the demo will use the info-at-kiwitcms.org email address for all communication sent. There are real people behind this address so if you write to us we will reply!

Changes since Kiwi TCMS 4.1.0

Enhancements

Bug fixes

Refactoring

Settings

Models and database migrations

API clients must now send values for estimated_time which must be in a format that parse_duration() understands, for example 'DD HH:MM:SS'! See https://docs.djangoproject.com/en/2.0/ref/utils/#django.utils.dateparse.parse_duration

IMPORTANT: this release introduces new database migrations!

Translations

There are new translatable strings introduced with this release! At this moment translating Kiwi TCMS in languages other than English is purely a community effort. If you'd like to help consult our translations contribution page.

How to upgrade

If you are using Kiwi TCMS as a Docker container then

cd Kiwi/
git pull
docker-compose down
docker pull kiwitcms/kiwi
docker pull centos/mariadb
docker-compose up -d
docker exec -it kiwi_web /Kiwi/manage.py migrate

Help us make Kiwi TCMS better

According to our 2018 roadmap the Kiwi TCMS team is investing a great deal of our time to make the code base more maintainable! For the past few releases we've been balancing code refactoring with bug-fixing and minor enhancements and we have lots of ideas down the line.

We want to hear your voice! Tell us how you use Kiwi TCMS inside your organization. We want to know what works for you and what doesn't so we can make it better. Feel free to comment below, file GitHub issues or send us an email.

Happy testing!

Kiwi TCMS 4.1.0

We're happy to announce Kiwi TCMS version 4.1.0! This is a bug-fix and enhancement update centered around providing more flexibility for administrators to configure the user approval process. Refer to the documentation for more information about signals handling in Kiwi TCMS.

Changes since Kiwi TCMS 4.0.0

Enhancements and bug fixes

Refactoring

Settings

Models and database migrations

IMPORTANT: this release introduces new database migrations!

Translations

There are new translatable strings introduced with this release, including translations of a few email templates! At this moment translating Kiwi TCMS in languages other than English is purely a community effort. If you'd like to help consult our translations contribution page.

How to upgrade

If you are using Kiwi TCMS as a Docker container then

cd Kiwi/
docker-compose down
docker pull kiwitcms/kiwi
docker-compose up -d
docker exec -it kiwi_web /Kiwi/manage.py migrate

Happy testing!

Kiwi TCMS & tcms-api version 4.0.0

We're happy to announce Kiwi TCMS version 4.0.0 and the corresponding tcms-api version 4.0.0! This is a very big release which introduces tons of backwards incompatible changes to the RPC layer as well as several bug-fixes and enhancements!

With this release Kiwi TCMS meets one of our milestones for 2018! All XML-RPC namespaces now match DB model names and the API client class names. Lots of similar and duplicate methods were removed and existing ones were simplified. We've also reduced code duplication and technical debth by 1 month according to stats by CodeClimate.

IMPORTANT: this is also the first release to officially publish the kiwitcms/kiwi image on Docker Hub!

WARNING: upgrading to Kiwi TCMS 4.0.0 means you will have to upgrade the tcms-api client as well. None of the 4.x releases is compatible with older releases due to the massive ammount of changes in the RPC layer.

Changes since Kiwi TCMS 3.50

Enhancements and bug fixes

Settings

RPC methods refactoring

Models and database migrations

IMPORTANT: this release introduces new database migrations!

tcms-api changes since 1.5.1

How to upgrade

If you are using Kiwi TCMS as a Docker container then

cd Kiwi/
docker-compose down
docker pull kiwitcms/kiwi
docker-compose up -d
docker exec -it kiwi_web /Kiwi/manage.py migrate

Happy testing!

Kiwi TCMS version 3.50

We're happy to announce Kiwi TCMS version 3.50 and the corresponding tcms-api version 1.5.1! This is a bug-fix and enhancement release and we advise you to upgrade as soon as possible!

Changes since Kiwi TCMS 3.49

Enhancements and bug fixes

Refactoring

tcms-api changes since 1.5.0

IMPORTANT: this release introduces new database migrations!

How to upgrade

If you are using Kiwi TCMS as a Docker container then

cd Kiwi/
docker-compose down
make docker-image
docker-compose up -d
docker exec -it kiwi_web /Kiwi/manage.py migrate

Happy testing!

Subscribe to our newsletter