Test Manager plugin for Trac

Jump to the list of the latest changes. and watch the Video tutorial on Youtube, helping old and new users familiarize with the changes and highlighting the recent features of new releases.

Follow this Quick setup guide to get you going quickly.

Note: TestManager release 1.4.8 introduces a new more robust upgrade mechanism.

Unfortunately upgrade to 1.4.8 or up is supported only from release 1.4.7 or later.. If you have an older version than 1.4.7, first upgrade to 1.4.7 and then you can upgrade to the latest version.

Note: TestManager release 1.4.3 requires XmlRpcPlugin to run. I've removed this hard dependency with 1.4.4. Anyway, if you have XmlRpcPlugin you can use the Test Manager remote API.

Description

A Trac plugin to create Test Cases, organize them in Catalogs, generate Test Plans and track their execution status and outcome.

Differently from other test management plugins for Trac that use Tickets as test case holders, this one uses Wiki pages and an additional proprietary data model to store Test Cases. This allows you to not pollute your ticket lists with something that is not a ticket, and at the same time is powered by the Trac search engine and formatting syntax for Wiki pages.

A set of plugins intercept requests for Wiki pages that are test cases and decorate the page with title, breadcrumbs, tree view, type-ahead search inside the catalogs, test case status semaphore and icons and buttons that allow you to create new test cases, sub-catalogs, copy and paste test cases around different catalogs and change a test case status.

Multiple Test Plans can be associated to any Test Catalog, in order to keep track of the execution of the corresponding Test Cases in a particular testing context.

A Programmatic API is available to interact with the test management environment with three types of interfaces:

• XML-RPC,
• HTTP RESTful and
• Python classes.

The plugin has been made modular:

• A generic, customizable Workflow Engine is provided as a separate plugin in the package, which allows you to associate custom workflows to any Resource in Trac, including Wiki pages and any custom object handled through a Resource Manager. This can be done very easily by adding as little as a few lines of code to the component which manages the User interface.

• All the data manipulation layer has been extracted into a Generic Persistent Class framework plugin, which can be easily used by any other plugin developer to provide persistence, security, change history, custom properties, pattern matching search, and other features to their basic data model with a handful lines of code.

All of the test objects, i.e. catalogs, test cases, test plans and test cases in a plan (i.e. with a status and a status change history), support:

• Custom properties, which can be declared in the trac.ini file and will be available to the User for change, stored in the database and available to change listeners.
• Change history
• Listener interface to be notified of object creation, modification and deletion
• Customizable Workflow state machine, declared in the trac.ini file, with the same syntax as for Ticket workflows (I may have reused some existing code here :-)
• Customizable Workflow Operations, via a plugin api so that any component can provide its custom operations to be performed upon any workflow action, as defined in the trac.ini file.
• Workflow also supports a listener API for components interested in state transitions and actions performed
• Workflow states also support custom properties, so to be able to convey additional context information on a workflow state and use it in listeners or directly from the database.

The developed workflow engine is able to work on any Trac Resource, it is not confined to this plugin ones. You can then define a workflow on any Trac resource, including Wiki pages, declaratively in the trac.ini file. You will then add a handful of custom code (for example in an ITemplateStreamFilter) to add the markup that the workflow engine generates for you to your desired Trac web page. See this page for further details.

The programmatic and the RESTful API are documented in details in this page TestManagerForTracPluginApi.

Here follows an overview of the plugin functionalities. For a brief tutorial, refer to the powerpoint presentations attached below. Note, anyway that the tutorial was developed for release 1.0.0 and so does not cover Test Plans, Workflows and Custom properties.

Test Catalogs

Test catalogs contain sub-catalogs or Test Cases. A Javascript tree view displays a catalog node and its sub-tree, including all of the test cases contained.

Next to each catalog (or sub-catalog) a number in brackets shows the number of test cases it contains.

Notice at the top of the page breadcrumbs to easily navigate up in the catalogs tree.

You can choose between a tree-like and a tabular representation of the catalog and all of its contained sub-catalogs and test cases, as shown in The former is useful to actually work on the Test Catalog, adding sub-catalogs and new test cases, or modifying them. The latter, which also reports a printout of all the Test Case descriptions, may be useful for a Test Engineer to save or print as a handout to actually run the test cases in the catalog.

You can add sub-catalogs or Test Cases simply by entering a name (blanks and case are supported) and click the appropriate button. A new wiki page is generated, with a naming convention allowing the plugin code to position it correctly in the catalogs tree, and opened for browsing.

If you wish to edit the title or add some textual contents to the (sub-)catalog, click on "Edit Page" at the bottom of the screen.

Be careful that the first line (the one surrounded by '==') will always be taken as the title of the catalog (the same stands for test cases, read below), so do not remove this line. You can edit this line to change the test catalog title, anyway.

Just save the page ("Submit Changes") to save your textual changes to the (sub-)catalog.

To delete a Test Catalog you must delete the corresponding Wiki page. Notice that this operation does not delete all of the Test Cases contained in the Catalog. You must either delete each Test Case individually, or first move them into a different catalog first.

If you mistakenly deleted a Test Catalog, you can save the day by recreating a Wiki page with the exact name. To do so, enter the desired name directly in the browser's URL, after the /wiki/ part. Trac will show you an empty page for the catalog, but already populated with all of your existing test cases and sub-catalogs. Click on "Create this page", give it a title (surrounded by '==') and submit your changes. The new (old) catalog is in place.

Tree View of the Test Catalog


Tabular view of the Test Catalog

Importing Test Cases from a CSV file into a Test Catalog

If you already have Test Cases defined elsewhere you can import them into the Test Manager. While browsing a Test Catalog, click on "Import Test Cases" at the bottom of the page. The dialog box shown in the following figure will pop-up. Browse for the CSV (comma-separated values) file containing your Test Cases, specify the value separator (default is the comma character) and click "Import".

The CSV file must have the following format.

The first row will have column names. Use lowercase identifiers, with no blanks, for them.

The data must start from the second row. There must be at least the following required columns:

• title
• description

Any subsequent columns are optional, and will generate custom test case fields. To fully appreciate custom fields, switch to the Test Catalog Tabular view.

A sample CSV file may be the following:

title,description,platform,greetings
First Test Case,This is the long description for the first test case,linux,bye bye
Second Test Case,This is the long description for the second test case,windows,ciao
Third Test Case,This is the long description for the third test case,macosx,hasta la vista

Import Test Cases dialog

Test Cases

Test Cases are the smallest units of test execution.

They are implemented again as wiki pages, with a naming convention that allows the plugin code to recognize them and treat them appropriately.

To add Test Cases into a Test Catalog, open the catalog, go to the bottom of the page and enter a name for the new Test Case to be created into the appropriate text box (blanks and case are supported). Then click the button "Add a Test Case". A new wiki page is generated, with a naming convention allowing the plugin code to position it correctly in the catalogs tree, and opened for editing. Be careful that the first line (the one surrounded by '==') will always be taken as the title of the Test Case, so do not remove this line. You can edit this line to change the test case title, anyway.

You can then add the Test §Case description just below the first line (i.e. title), using WikiFormatting, adding attachments and everything else is supported for Wiki pages.

When you are done, save the page ("Submit Changes") to save your new Test Case.

To delete a Test Case you must delete the corresponding Wiki page.

Again, notice the breadcrumbs at the top, useful to go back to the enclosing catalog or any catalog up the hierarchy.

Moving or Copying Test Cases from one catalog to another

It is also possible to move a single Test Case, or copy multiple Test Cases into a different catalog, with an experience similar to cut&paste.

Moving a single Test Case

To move a single Test Case from one catalog into a nother, you:

1. Open the Test Case
2. Click on the "Move the Test Case into another catalog" button. This is similar to a "cut" operation.
3. Navigate and open the destination catalog
4. Click on "Move the copied Test Case here" button (which only appears if a Test Case has been cut first).

It is also possible to cancel the operation at any time by clicking the "Cancel" button in a Gmail-type of yellow message at the top of the page.

Moving a single Test Case

Copying multiple Test Cases

To copy one or more Test Cases, even from different catalogs, and paste them all together into a destination (sub-)catalog, you:

1. Open the Catalog. Expand the tree nodes as you like.
2. Click on the "Select Multiple Test Cases" button. A checkbox appears next to every Test Case in the catalog and the sub-catalogs. See the next figure.
3. Select the Test Cases you wish to copy into another catalog.
4. Click on the "Copy the Selected Test Cases" button. This is similar to the usual "Copy" operation.
5. Navigate and open the destination catalog, or create a new one.
6. Click on "Paste the copied Test Cases here" button (which only appears if Test Cases have been copied first).

Note that the test execution statuses (i.e. successful, failed, etc...) associated to the original Test Cases in any Test Plan are not copied along with them.

Copying multiple Test Cases

Test Plans

Since: 1.1.0

A Test Plan represents a plan for a particular execution of all the Test Cases in a Test Catalog (or sub-catalog).

Think for example at the build verification test following a nightly build, or, for traditional projects, Technical Test and eventually Client Test.

Thus a Test Plan is associated to one Test Catalog, or sub-catalog. You can have any number of Test Plans for one Test Catalog, anyway. The list of Test Plans you generated for a Test Catalog is displayd in a table at the bottom of the same catalog, as shown in the following figure.

To create a Test Plan for a catalog, open the desired Test Catalog (or sub-catalog), enter the name of the new Test Plan in the appropriate test box and click "Generate a new Test Plan".

The new Test Plan will be opened for display, showing all of the Test Cases in the catalog, in the "Untested" status.

As with Test Catalogs, you can choose between tree-like and a tabular representation of the Test Plan, as shown in the figures below. The former is useful to actually work on the Test Plan and to view test cases before actually going and executing them. The latter, which indicates for each Test Case the latest status modification and its author, may be useful to keep as the documentation of a just run test plan.

Tree View of the Test Plan


Tabular view of the Test Plan

You have two ways to set the execution status of a Test Case in a particular Test Plan:

1. Directly from the test plan tree view, by clicking on the stop-light icon and choosing the new color from the menu that opens (see next figure). (Since 1.4.7)
2. By opening the test case first, clicking on its name in the Test Plan tree, and then clicking on the corresponding stop-light at the bottom of the page, as shown in the following figure.

You don't need to save anything, the changes are immediately recorded in the database and reflected in the Test Plan.

Custom Test Case Outcomes (i.e. states, verdicts)

Since: 1.4.1

You can customize the Test Case outcomes (i.e. states, verdicts) to match your internal quality process.

The default outcomes are configured in trac.ini at plugin installation (or upgrade) time as follows:

[test-outcomes]
green.successful = Successful
red.failed = Failed
yellow.to_be_tested = Untested
default = to_be_tested

You can customize the outcomes at any moment by adding new outcomes, or modify the descriptions of the current ones. Do not delete previous outcomes if you have already assigned them to any of your test cases.

For example, to add a new outcome named "It's a Mess!!!" to the failures, add a line like the following, in trac.ini:

red.bigmess = It's a Mess!!!

You will now be able to choose it when assigning a test case status using the semaphore, which now shows a more complete menu to choose from. See the following figure.

Linking a Ticket to a Test Case

When viewing a Test Case, you can open a new Ticket by means of the "Open Ticket on this Test Case" button.

The new ticket will contain a link back to the corresponding Test Case and, if you were viewing it in the context of a particular Test Plan, of the Test Plan as well. Additionally, a programmatic Python API is provided to retrieve all of the Tickets opened against a Test Case, in all or in a specific Test Plan.

This plugin also supports the TracTicketTemplatePlugin to fill a ticket template with this information. In this case, you can use the following parameters in the template to receive the information:

• testCaseNumber: The wiki page for the corresponding Test Case
• planId: The ID of the Test Plan
• planName: The name of the Test Plan

For example, to get the test case number, you template will have something like:

  bleah bleah
  Test Case: %(testCaseNumber)s
  bleah bleah

Showing Tickets related to a Test Case

While browsing a Test Case, either inside or outside of a Test Plan, you can show all of its related Tickets - i.e. Tickets that were opened using the above "Open a Ticket on this Test Case" button.

To do this, just browse the Test Case and click on the "Show Related Tickets" button.

If you are browsing a Test Case "definition", i.e. from a Test Catalog, you will be shown all of the Tickets opened against the Test Case in any Test Plan.

If you are instead browsing a Test Case in a specific Test Plan, you will be shown all of the Tickets opened against the Test Case in that specific Test Plan.

See the following figure for what happens when you click on the "Show Related Tickets" button from the Test Case shown at the previous section.

Exporting Test Catalogs or Test Plans into CSV file

Since: 1.4.9

You can export to CSV format a Test Catalog or a Test Plan.
In the Test Catalog and Test Plan pages you will find an "Export ..." button.
The export format is designed for you to elaborate your external statistics, but also to eventually import the stuff back into another Test Manager plugin evironment, as soon as a compatible import feature will be implemented :D (the current import feature is simpler than that).
In the export dialog dhown in the next figure, you can specify whether the output should contain the complete text of the Test Cases, and whether this should be pre-formatted (WYSIWYG) or in raw WikiFormatting syntax.

In addition to that, you can specify which column separator to use in the exported CSV file.

Custom fields

Since: 1.2.0

Custom fields can be added to the four test objects and to the workflow state object, by declaring them in the trac.ini file.

The syntax is the same used for custom Ticket properties, only the name of the ini file sections are specific: you must use the test artifact type name followed by "-tm_custom".

The test artifacts type names are the following:

• testcatalog
• testcase
• testplan
• testcaseinplan

For example, the following sections in the trac.ini file define one custom property for each of the above artifacts.

    [testcatalog-tm_custom]
    prop1 = text
    prop1.value = Default value
    prop1.label = Property One

    [testcaseinplan-tm_custom]
    os = text
    os.value = windows
    os.label = Operating System

    [testcase-tm_custom]
    nice_prop = text
    nice_prop.value = My friend

    [testplan-tm_custom]
    good_prop = text
    good_prop.value = linux

Note: Only text property types are currently supported.

Once defined in the trac.ini file as above, custom fields will be available to the User for browse and for editing in the Web page, as shown next.

Note: Editing custom properties requires the TEST_MODIFY permission.

The following screenshot shows a custom "platform" field added to the Test Case artifact, and how it is presented to the User for editing.

The value is initially displayed read-only, as a label. Clicking on the pencil icon turns the label into an edit box, allowing the User to edit the value and also displays a "Save" button. Clicking the button immediately saves the new value into the database.

Customizable Workflow

Since: 1.2.0

The following figure shows a sample workflow added to Test Cases with custom sample operations. No built-in operation is currently implemented but the sample one shown here, named 'sample_operation', which logs a debug message with the text input by the User.

Every object which has a workflow defined is created in a "new" state, so every transition should consider this as the first state in the state machine.

This is a sample content of the trac.ini file to associate a workflow to the Test Case object.

[testcase-resource_workflow]
sleep = new -> asleep
sleep.permissions = TEST_MODIFY

sing = new -> singing
sing.permissions = TEST_MODIFY
sing.operations = sample_operation

calmdown = singing -> calm
calmdown.permissions = TEST_MODIFY

kill = asleep,calm -> dead
kill.permissions = TEST_MODIFY

Refer to the Workflow Engine description page for more details.

Searching and filtering Test Cases in the tree view

A type-ahead, browser-side filtering feature allows for easily locating Test Cases matching a particular naming convention in the title, and/or a particular execution status.

This is available both in the context of a Test Catalog and in Test Plans.

Multiple words (or parts of) separated by blanks are supported, in AND condition.

In the case of a Test Plan, you can also add a test case status to filter by this criterion. The supported statuses are (even substrings of):

• untested
• successful
• failed

Test Artifact Change History

Since: 1.4.8

On each test artifact page you will find a section named "Object change history", which contains a list of all the changes to any standard and, more interestingly, custom object's property.

In the following figure you can see how the "device" custom property has been changed in a Test Plan history.

Object change history for a Test Plan

Test Management and Execution Statistics

Since: 1.1.1

Charting capabilities allow for tracking the evolution of the test suites, the corresponding test plans and tickets opened against test cases.

To access the test management statistics, click on Test Stats in the Trac toolbar on the upper right corner of the page.

As shown in the next figure, a chart will be displayed with statistics about all the test cases in the system and a default period of time, with more charts available in the other tabs.

There are three charts available:

Test activity trend chart

Displays the number of test cases created during time and the trend of the test activity, in terms of test cases to be tested, successful and failed.

By means of the filtering criteria at the bottom of the chart, you can select the desired period of time, the chart resolution (in terms of time between different samples) and the Test Plan for your chart.

You can bookmark the URL named "Static URL" at the bottom of the page in order to go directly with the current selected filtering criteria.

Test activity trend chart

Current test status chart

Shows the current status of the test activity for all test plans of a specific plan, displaying the number of tests that are successful, failed and yet to be tested.

Current test status chart

Tickets trend chart

As long as tickets have been opened from the context of a test case in a plan, by means of the appropriate "Open a ticket on this test case" button, this chart shows the trend in tickets associated with all test plans or a specific plan, displaying the number of total, active (i.e. not closed) and closed tickets.

Tickets trend chart

Configuration

Note: By default, the charts use the YUI library available online at Yahoo!, so you will need internet connection from your client machines to be able to display this charts.

Alternatively, in case you don't have internet connection, you can download the YUI library, put it somewhere on the Trac server machine where the web server can access it, and refer its location in the trac.ini as follows.

Other options you can customize are the default period of time the charts will display, in terms of historical depth and scale.

[testmanager]
yui_base_url = <URL or path to the root of your YUI library>
default_days_back = <number of days back from today>
default_interval = <number of days between samples>

For example:

[testmanager]
yui_base_url = http://yui.yahooapis.com/2.9.0
default_days_back = 14
default_interval = 1

Security

The following new permissions are available to manage the Test Manager security:

• TEST_VIEW - Ability to view test catalogs and test cases
• TEST_MODIFY - Ability to create and edit test catalogs and test cases
• TEST_EXECUTE - Ability to change the status of a test case in a test plan
• TEST_DELETE - Ability to delete test cases
• TEST_PLAN_ADMIN - Ability to generate and delete test plans
• TEST_STATS_VIEW - Ability to view test management statistics

Programmatic API

The programmatic RESTful API is documented in details in this page TestManagerForTracPluginApi.

Tutorial (as powerpoint presentation)

TODO: These tutorials document release 1.0.3 and so need to be updated with the Test Plan and other recent features.

• attachment:"UserGuide_part_1.ppt"
• attachment:"UserGuide_part_2.ppt"
• attachment:"UserGuide_part_3.ppt"

Project site

Project site: http://sourceforge.net/projects/testman4trac/

The project is also on Pypi: http://pypi.python.org/pypi/TestManager

Translations

This plugin is localized and all texts and messages will be displayed in the language detected by the User's browser, if available, or in English otherwise.

You may want to do your own translation, following the localization cookbook page for Trac plugins.

You've done a new translation? Superb! Contributing your translation is highly appreciated.

Contact me on the forum and I'll manage to add it to the set of available languages.

Top translations » Test Manager for Trac Plugin on Transifex

Kindly provided by

Bugs/Feature Requests

If you have any issues, create a new ticket.

Existing bugs and feature requests for TestManagerForTracPlugin are here.

#7479

Attachments on the TestManagerForTracPlugin page

#7483

Avoid changing test result with one click, add a confirmation button

#7484

Reverse the order of changes in the test case status change history

#7485

"Open ticket on this test case" should work without a patched TracTicketTemplatePlugin

#7486

Customizable test case outcomes (aka verdicts)

#7487

Add multiple test plans capability

#7506

Implement security permissions

#7507

Implement security permissions

#7526

Add ability to duplicate a test case

#7536

Add test management statistics

#7551

Test statistics don't work on Trac 0.11

#7552

Export test statistics in CSV and bookmark this chart features in the test stats chart

#7569

Add listener interface to let other components react to test case status change

#7570

Add a relationship table between tickets and test cases in plan, and corresponding API

#7704

Add workflow capabilities, with custom states, transitions and operations, and state transition listeners support

#7705

Add support for custom properties and change history to all of the test management objects

#7726

"Perform Action" button appears even if no workflow is defined for an object

#7780

Display in tabular views also any custom property specified on catalogs, plans or test cases

#7781

Add ability to filter test cases also in the tabular view, and to consider custom properties

#7782

Full Test Plan display/print

#7845

Restore compatibility with Trac 0.11

#8004

Cannot search if an admin

#8006

Add search for test plans in global Trac search page

#8007

Add out-of-the-box operations for use in custom workflows

#8019

Ability to duplicate an entire test catalog under a new name with one button click

#8049

Add ability to delete a Test Plan

#8050

Add ability to delete a Test Plan

#8084

ordering issue

#8121

Catalog Wiki Page not added

#8123

Can't move testcase more than one time into different catalog

#8124

AttributeError: 'NoneType' object has no attribute 'splitlines'

#8125

"duplicate test case" does not work for previously moved test case

#8127

Ability to move/link multiple test cases to an other catalog

#8144

Test statistical charts don't show successful and failed figures

#8154

LookupError: unknown encoding: cp0

#8259

Add navigation from a test case to its related tickets

#8261

Error: Requested field information not found for class resourceworkflowstate.

#8321

Add standard internationalization support (i18n)

#8322

Show timestamps according to User's locale

#8323

Trac 0.11 - Unable to expand Available plans and Test case status change history collapsable sections

#8328

testplan enhancements

#8378

Set date and time format correctly in Test Stats page

#8392

Provide ticket subject automatically when opening ticket on failed test case

#8427

Add XML-RPC complete interface for remote management of test objects

#8560

[TestManager 1.4.3] Parse error thrown everytime after trying to create a new catalogue

#8567

javascript error when deleting test plans

#8596

Remove hard dependency on XML RPC plugin for Trac 0.11

#8613

TestManager is not installable on Trac 0.12.2

#8671

Installation problem on Trac 0.11 and PosgreSQL

#8672

Installation problem on Trac 0.11 and MySQL

#8727

installation problem with 0.12

#8735

Update the powerpoint tutorials

#8761

Copy multiple test cases into another catalog

#8765

trac killed after installation of the 3 plugins

#8815

Ability to delete multiple test cases

#8816

Ability to cut, copy or delete an entire (sub)catalog

#8825

Ability to import test cases from Excel (CSV) file

#8845

Don't see "Test Manager" tab

#8869

Loading of Test Manager takes too long and sometimes time out

#8871

No # allowed at custom fields

#8872

tracing of (requirement) tickets at test cases

#8873

css styles ar not compatible with the agilo plugin

#8874

Next test case button

#8875

More visibility for Tickets related to test suites

#8876

Can't create Catalogs/Test cases when trac runs from site root

#8878

TestManagerForTracPlugin does not play well with MenusPlugin

#8898

yui_base_url not honored in templates ?

#8900

Add ability to articulate test plans on two dimensions, e.g. test cases and platforms

#8907

Add template for "New TestCase"

#8908

Possiblity to change test case status from the tree view

#8913

NameError: global name 'NEXT_PROPERTY_NAME' is not defined creating a new catalog or test case

#8931

Problem with Attatchemnts in test Plan

#8932

Reports

#8958

An ability to export test data to CSV format needed

#8985

fail on fresh install CentoOS 5.6

#9007

I can not import test cases

#9041

Test Manager installation

#9044

csv import not possible with agilo plugin

#9045

custom fields in ie8 not editable

#9049

comments for each testcase at testplan

#9077

Ability to separate and report on test plans by product

#9089

way to eliminate massive test cases?

#9135

Spanish translation testmanagerForTracPlugin

#9141

Update installation 1.4.6 -> 1.4.7 not possible

#9167

installation of 1.4.7 with postgres database not possible

#9187

Current test status report should consider only last result of a testcase in the plan

#9208

Test plan with only selected test cases from the catalog, take snapshot version of test cases

#9213

Test Management Statistics

#9287

new browser tab for existing testcases

#9288

CamelCase word at testcase/catalog name

#9297

can't print testplan table overview

#9510

TestStats for single Testplans at 1.4.8 not working - patched

#9530

Expand all / Collapse all is not running

#9614

Permission - no full admin

#9654

Error when creating test plans

#9663

Stats - Current test status for All Tests wont fit

#9672

TestManager Fails with MySQL

#9725

Quick statut changer in Test Plan not appears in Google Chrome

#9751

Ability to sort catalogs

#9754

Setting test result seemingly succeeds with expired login

#9758

Can't delete Test Case in v1.4.9

#9776

Testplans not visible with latest Agilo Plugin

#9857

Deleted test plan is still shown in Test Stats

#9865

Corrupt database after activation of Plugin

#9914

Test plan report that lists failed Test Cases along with their defect Tickets

#9950

easy_install not available

#9953

Changing a testcase custom field value gives and error

#10043

Deleting "zombie" TestPlans

#10071

Do not use apostrophes in test case names

#10077

upgrade from 1.4.6 to 1.4.7 has database transaction error

#10131

ProgrammingError: operator does not exist: text = integerLINE 1: DELETE FROM testcaseinplan WHERE id = 398 ^HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.

#10217

Deleting a Test Case, deleting Test Catalogs containing Test items

#10271

description is empty in tabular view

#10293

New install impossible on Trac 1.0beta1

#10294

New install impossible on Trac 1.0beta1

#10295

Trac detected an internal error: UnicodeError: source returned bytes, but no encoding specified

#10529

Sorting via internal ID

#10566

Zombie test catalogs in template management page

#10567

Zombie test catalogs in template management page

#10568

Can't edit test case templates

#10569

Can't delete unused test case templates

#10570

Link test cases to components

#10571

Wrong date format and missing date picker in test stats

#10578

Display Path for TestCaseInPlan

Download

Download the egg files, ready to be installed, from the project site here.

Source

You can download the source code from here.

Installation

Both Trac 0.11 and 0.12 are supported, and also Python 2.5, 2.6 and 2.7.

There used to be known issues on Trac 0.11 with MySQL and PosgreSQL (caused by lack of Trac support to the "int64" database column type) until release 1.4.6, but now they are fixed.

Note: If you are using MySQL or PosgreSQL, the database user must have read access to the trac.ini.

The plugin can be installed as usual, from the Trac administration panel, and requires a database upgrade.

You can follow the instructions in the INSTALL.txt file contained in the binary packages.

The functionalities are divided in three plugins, which must be installed in this order:

1) Trac Generic Class

2) Trac Generic Workflow

3) Test Manager

An additional plugin is only useful for debugging and should not be installed in a production environment.

• SQL Executor

With the latter plugin, you can perform arbitrary SQL queries on the Trac database from a web browser. Just type on the URL:

http://yourserver/yourenvironment/sqlexec?sql=any SQL statement

For example:

http://localhost/sampleproject/sqlexec?sql=select * from testcatalog

Follow the tutorial presentation that you can download from the project site to start. The tutorial is actually very old and needs to be updated to the new Test Plans and other features, but it will get you started.

Troubleshooting

Is setuptools properly installed?

Try this from the command line:

$ python -c "import pkg_resources"

If you get no output, setuptools is installed. Otherwise, you'll need to install it before plugins will work in Trac.

Did you get the correct version of the Python egg?

Python eggs have the Python version encoded in their filename. For example, MyPlugin?-1.0-py2.4.egg is an egg for Python 2.5, and will not be loaded if you're running a different Python version (such as 2.6 or 2.7).

Also, verify that the egg file you downloaded is indeed a ZIP archive and not the HTML preview page instead.

Is the plugin enabled?

If you install a plugin globally (i.e. not inside the plugins directory of the Trac project environment) you will have to explicitly enable it in trac.ini. Make sure that:

• you actually added the necessary lines to the [components] section:

  [components]
  tracgenericclass.* = enabled
  tracgenericworkflow.* = enabled
  testmanager.* = enabled
  

• the package/module names are correct
• the value is "enabled", not e.g. "enable"

Check the permissions on the egg file and on the trac.ini file

The user user to run Trac (either tracd or Apache) must be able to read the egg files.

Also, it's been reported that for PosgreSQL also its user must be able to read trac.ini.

Check the log files

Open the Trac Admin tab, enable logging and set the log level to DEBUG, then watch the log file for messages about loading plugins.

Verify you have proper permissions

Some plugins, including the Test Manager ones, require you have special permissions in order to use them.

See TestManagerForTracPlugin#Security for a description of the available/required permissions.

Is the wrong version of the plugin loading?

If you put your plugins inside plugins directories, and certainly if you have more than one project, you need to make sure that the correct version of the plugin is loading. Here are some basic rules:

Only one version of the plugin can be loaded for each running Trac server (ie. each Python process). The Python namespaces and module list will be shared, and it cannot handle duplicates. Whether a plugin is enabled or disabled makes no difference.

A globally installed plugin (typically setup.py install) will override any version in global or project plugins directories. A plugin from the global plugins directory will be located before any project plugins directory.

If your Trac server hosts more than one project (as with TRAC_ENV_PARENT_DIR setups), then having two versions of a plugin in two different projects will give uncertain results. Only one of them will load, and the one loaded will be shared by both projects. Trac will load the first found - basically from the project that receives the first request.

Having more than one version listed inside Python site-packages is fine (ie. installed with setup.py install) - setuptools will make sure you get the version installed most recently. However, don't store more than one version inside a global or project plugins directory - neither version number nor installed date will matter at all. There is no way to determine which one will be located first when Trac searches the directory for plugins.

Have you restarted Apache?

For Apache to sense a plugin upgrade (and sometimes even installation), it must be restarted.

If all of the above failed

OK, so the logs don't mention plugins, the egg is readable, the python version is correct and the egg has been installed globally (and is enabled in the trac.ini), you restarted Apache and it still doesn't work or give any error messages or any other indication as to why?

Open a ticket here. The plugins maintainers (yes, it's me :-)) watch ticket notifications continuously.

Recent Changes

Note: you can subscribe to the Forum to be updated on new releases.

During the last months:

Release 1.5.2 (2012-10-14):

This release makes the plugins compatible with Trac 1.0.

Since the release of Trac 1.0, I had to split the plugins code into two versions:

o A version that works well with Trac 0.11 o A version that works well with Trac 0.12 and Trac 1.0

All the code still works with Python 2.5, 2.6 and 2.7. All database backends are still supported.

To build from the source, refer to the updated BUILD.txt file.

o Fixed Ticket #10293 (Track-Hacks): New install impossible on Trac 1.0beta1

o Fixed Ticket #10295 (Track-Hacks): Trac detected an internal error: UnicodeError?: source returned bytes, but no encoding specified

Release 1.5.1 (2012-08-12):

This is somewhat of a major release, in that it includes several new interesting features and many bug fixes. See the list below for more details.

In addition to that, I realized a VIDEO TUTORIAL showing out the main features of this Test Manager, to help new and existing users take confidence with recent changes, new features and so on.

Check the tutorial on YouTube?: http://www.youtube.com/watch?v=BIi3QMT0rT4

Also the trac-hacks user manual has been restructured, updated and enriched with the recent changes and features.

o Enhancement #353771 (SourceForge?): Time tracking capability.

This is now possible by means of the enhancements to the tabular views of both test catalogs and test plans and the addition of a custom field. See the comments to the feature request and the video tutorial, where this customization example is fully explored.

o Enhancement #3537704 (SourceForge?): Possibility to select columns for the table view.

This is now possible from the administration panel.

I also added an interesting feature that allows for adding a statistical row to the test catalog and test plan tabular views, with the option to have the sum, average or count of the values in any column, being it a standard or a custom property of test cases or catalogs.

This feature is also used to realize the feature #353771 above. See the video tutorial for an interesting use of this one to track test execution effort estimation of an entire test plan and compare it to actual execution time.

o Enhancement #3537700 (SourceForge?): Option to select table view as default view.

This is now possible from the administration panel.

o Enhancement #3537696 (SourceForge?): Possibility to change test plans.

This is now possible by means of several new actions available on test cases:

• On a test case definition page, you have a new action button to add the test case to a plan. A dialog box appears showing you the suitable plans (the ones not containing all test cases).
• On a test case instance (a test case inside a plan) page, you have a new action button to remove the test case from the plan.
• On a test case instance (a test case inside a plan) page, in case the plan is containing an old snapshot of the test case and you wish to update its text description to the latest version, you have a new action button to do that.
  

o Fixed Ticket #8932 (Track-Hacks): The test plan tabular view has been enhanced to also

show test case full text description

o Fixed Ticket #10131 and #10217 (Track-Hacks): Deleting a Test Case, deleting Test Catalogs containing Test items.

This only happened on PosgreSQL.

o Fixed some bugs that I found ad which were not reported:

• The quick search in test catalogs was not working.
• Indentation of test cases in tabular views for both catalogs and plans was incorrect.
• Localization (i.e. translation) did not work for some parts of the Administration panel, the Statistical charts page and several dialog boxes.
• Setting a test case status using the pop-up semaphore menu did not work on test plan tabular view.
• Some more I can't recall :D
  

Release 1.4.11 (2012-06-03):

The data referential integrity in case of test catalog, test case and test plan deletion has been enhanced.
Deleting a test catalog now also deletes all of the contained sub-catalogs, test cases and test plans, as well as the corresponding status change history.

o Fixed Ticket #9857 (Track-Hacks): Deleted test plan is still shown in Test Stats
o Fixed Ticket #9953 (Track-Hacks): Changing a testcase custom field value gives and error
o Fixed Ticket #10043 (Track-Hacks): Deleting "zombie" TestPlans?

Release 1.4.10 (2012-03-03):

o Enhancement #9751 (Track-Hacks): Ability to sort catalogs.

Test Catalogs are now sorted by title in the tree and table views.

o Fixed Ticket #9776 (Track-Hacks): Testplans not visible with latest Agilo Plugin.
o Fixed Ticket #9530 (Track-Hacks): Expand all / Collapse all is not running.

This only happened with the Agilo plugin installed.

o Fixed Ticket #9754 (Track-Hacks): Setting test result seemingly succeeds with expired login.

Now operations such as setting a test case status and updating a custom field,
when failing will display a dialog box with an error message.

o Fixed Ticket #9758 (Track-Hacks): Can't delete Test Case in v1.4.9.

Release 1.4.9 (2012-01-04):

o Enhancement #8958 (Track-Hacks): An ability to export test data to CSV format needed.

You can now export to CSV a test catalog or a test plan.
In the test catalog and test plan pages you will find an "Export ..." button.
The export format is designed to eventually import the stuff back into another Test Manager plugin evironment, as soon as a compatible import feature will be implemented :D (the current import feature is simpler than that.)

o Enhancement #9287 (Track-Hacks): New browser tab for existing testcases. Now you have an option in trac.ini to choose what

to do when clicking on a test case in the catalog/plan.
The default is to NOT open it in a new window, but you can change that from the admin panel,
under Testmanage->Settings, or directly in the trac.ini file like this:

[testmanager]
testcase.open_new_window = True

o Fixed Ticket #9297 (Track-Hacks): Can't print testplan table overview. There was a problem with the default print css from Trac.
o Fixed Ticket #9510 (Track-Hacks): TestStats? for single Testplans at 1.4.8 not working - patched.

Thanks so much Andreas for finding it and for patching it!!!

o Fixed Ticket #9530 (Track-Hacks): Expand all / Collapse all is not running
o Fixed Ticket #9654 (Track-Hacks): Error when creating test plans

Release 1.4.8 (2011-10-23):

o Strongly enhanced the upgrade mechanism. Now it's more robust, should work with all the databases and between arbitrary Test Manager versions.

The only drawback is that upgrade is only supported from 1.4.7, not from previous versions.

o Enhancement #9077 (Track-Hacks): Ability to separate and report on test plans by product
o Enhancement #9208 (Track-Hacks): Test plan with only selected test cases from the catalog, take snapshot version of test cases.

This is an important one. Many users were asking for a way of including only selected test cases into
a Test Plan, for different reasons. Now you have it :D

o Added French language catalog! Thanks to someone who doesn't want to be cited :D
o Added Object change history view. It shows the change history of standard and, more interestingly, custom properties of any test artifact. o Fixed Ticket #9141 (Track-Hacks): Update installation 1.4.6 -> 1.4.7 not possible
o Fixed Ticket #9167 (Track-Hacks): installation of 1.4.7 with postgres database not possible
o Fixed Ticket #9187 (Track-Hacks): Current test status report should consider only last result of a testcase in the plan.

Thanks to Andreas for his contribution to fixing this one!

Release 1.4.7 (2011-08-28):

o Enhancement #8907 (Track-Hacks): Add template for "New TestCase?" - Thanks a lot to Christian for the hard work on this enhancement!!!

Now you can define templates for your new test catalogs and new test cases, and assign default templates based on each test catalog!

o Enhancement #8908 (Track-Hacks): Possiblity to change test case status from the tree view.

No more need to open each test case in a plan to set its result, you can now do this directly from the tree view!

o Fixed Ticket #8869 (Track-Hacks): Loading of Test Manager takes too long and sometimes time out
o Added Spanish and German catalogs! Thanks a lot to Christian and Andreas for the translations!!! Italian was already part of the plugin.

Release 1.4.6 (2011-06-19):

o Fixed Ticket #8871 (Track-Hacks): No # allowed at custom fields
o Fixed Ticket #8873 (Track-Hacks): css styles ar not compatible with the agilo plugin
o Fixed Ticket #8876 (Track-Hacks): Can't create Catalogs/Test cases when trac runs from site root
o Fixed Ticket #8878 (Track-Hacks): TestManagerForTracPlugin does not play well with MenusPlugin
o Fixed Ticket #8898 (Track-Hacks): yui_base_url not honored in templates ?
o Enhancement #8875 (Track-Hacks): More visibility for Tickets related to test suites
Added more statistical charts, including Current test status pie chart and Tickets related to test cases trend chart
Simplified setting the outcome of a test case

Release 1.4.5 (2011-05-21):

o Enhancement #8825 (Track-Hacks): Ability to import test cases from Excel (CSV) file

Release 1.4.4 (2011-03-11):

o Fixed Ticket #8567 (Track-Hacks): Javascript error when deleting test plans
o Enhancement #8596 (Track-Hacks): Remove hard dependency on XML RPC plugin for Trac 0.11
o Enhancement #8761 (Track-Hacks): Copy multiple test cases into another catalog
Added wiki documentation for copying multiple test cases into another catalog.

Release 1.4.3 (2011-01-20):

o Enhancement #8427 (Track-Hacks): Add XML-RPC complete interface for remote management of test objects
Added wiki documentation for the new XML-RPC API.

Release 1.4.2 (2011-01-09):

o Fixed Ticket #8378 (Track-Hacks): Set date and time format correctly in Test Stats page
Also added support for custom test case outcomes in the Test Stats page
Added Wiki documentation on the ability to customize the test case outcomes (i.e. states, verdicts) and to link Tickets to Test Cases.

Release 1.4.1 (2010-12-27):

o Enhancement #7486 (Track-Hacks): Customizable test case outcomes (aka verdicts)
o Enhancement #7570 (Track-Hacks): Add a relationship table between tickets and test cases in plan, and corresponding API
o Added wiki documentation on the Generic Persistent Class framework

Release 1.3.12 (2010-12-19):

o Enhancement #8321 (Track-Hacks): Add standard internationalization support (i18n)
o Enhancement #8322 (Track-Hacks): Show timestamps according to User's locale
o Fixed Ticket #8323 (Track-Hacks): Unable to expand Available plans and Test case status change history collapsable sections

Release 1.3.11 (2010-12-02):

o Added out of the box operation to workflow engine: set_owner and set_owner_to_self
o Enhancement #8259 (Track-Hacks): Add navigation from a test case to its related tickets

Release 1.3.10 (2010-11-28):

o Fixed Ticket #8154 (Track-Hacks): LookupError?: unknown encoding: cp0

Release 1.3.9 (2010-11-23):

o Fixed Ticket #8144 (Track-Hacks): Test statistical charts don't show successful and failed figures

Release 1.3.8 (2010-11-23):

o Fixed Ticket #8121 (Track-Hacks): Catalog Wiki Page not added
o Fixed Ticket #8123 (Track-Hacks): Can't move testcase more than one time into different catalog
o Fixed Ticket #8124 (Track-Hacks): AttributeError?: 'NoneType?' object has no attribute 'splitlines'
o Fixed Ticket #8125 (Track-Hacks): "duplicate test case" does not work for previously moved test case

Release 1.3.7 (2010-11-20):

o Enhancement #7704 (Track-Hacks): Add ability to delete a Test Plan
o Fixed Ticket #8084 (Track-Hacks): Ordering issue

Release 1.3.6 (2010-11-09):

o Fixed Ticket #8004 (Track-Hacks): Cannot search if an admin

Release 1.3.5 (2010-10-17):

o Restored compatibility with Trac 0.11. Now again both 0.11 and 0.12 are supported.

Author/Contributors

Author: seccanj
Maintainer: seccanj
Contributors: