enfold.recipe.funkload

Funkload Recipe

The funkload recipe, enfold.recipe.funkload, creates funkload scripts for a project. XXX

The funkload recipe has several options:

python

The python interpreter path. This option is mandatary.

working-directory

The working-directory option lets to specify a directory where funkload the tests will run (i.e : where funkload tests are defined. Funkload will change to this directory when run. If the working directory is the empty string or not specified at all, the recipe will create a working directory among the parts.

server-url

The base server url on which the funkload tests will be run against. Here is an example : http://localhost:8080 In case of Zope dedicated buildout, note you can use variable substitions such as : server-url=http://localhost:${z2:instance:http-address} where z2-instance is the name of the buildout part defining a Zope2 instance. If not specified then it will use the server_url variable from the main section of the funkload configuration file inside working-directory.

test-script

The name of the Funkload test runner script that the recipe will generate inside the buildout bin directory. Defaut value is "fl-tests"

bench-script

The name of the Funkload bench runner script that the recipe will generate inside the buildout bin directory. Default value is "fl-bench"

monitor-script

The name of the Funkload monitor controller script that the recipe will generate inside the buildout bin directory. Default value is "fl-monitor"

creds-script

The name of the Funkload credential runner script that the recipe will generate inside the buildout bin directory. Default value is "fl-creds"

record-script

The name of the Funkload recorder script that the recipe will generate inside the buildout bin directory. Default value is "fl-record"

report-script

The name of the Funkload bench report builder script that the recipe will generate inside the buildout bin directory. Default value is "fl-report"

start-runtime-deps

Sequence of runtime service deps scripts that need to be started before Funkload tests are launched.

Imagine you are running Funkload tests against Zope, you may specified the zopectl path here so that it starts the Zope instance and a ZEO server automatically.

For example

start-runtime-deps =
    [ ('${buildout:bin-directory}/${zopepy:interpreter}',
      '${buildout:directory}/bin/zeo' ,
      ('start',)),
      ('${buildout:bin-directory}/${zopepy:interpreter}',
      '${buildout:directory}/bin/zope' ,
      ('start',)),
    ]

Note, this is will be evaluated as a Python expression. See quotes around variables for instance.

dsn

The database connection string for the database to populate with Funkload's results.

passing-conditions

A list of dicts that define the minimum acceptable performance. Each dict specifies the Funkload test's module, class, and method, as well as for some number of concurrent users, the minimum acceptable number of requests served per second. For the same test module, class, and method, it's acceptable to define multiple performance criteria for differing numbers of concurrent users.

For example

passing-conditions =
    [
        {
            'module': test_enfold_server,
            'klass': TestEnfoldServer,
            'method': test_homepage_anonymous,
            'concurrent_users': 20,
            'reqs_per_sec': 15,
        },
    ]

stop-runtime-deps

Sequence of runtime service deps scripts that need to be stoped before Funkload tests are launched.

For example

stop-runtime-deps =
    [ ('${buildout:bin-directory}/${zopepy:interpreter}',
      '${buildout:directory}/bin/zeo' ,
      ('stop',)),
      ('${buildout:bin-directory}/${zopepy:interpreter}',
      '${buildout:directory}/bin/zope' ,
      ('stop',)),
    ]

Note, this will be evaluated a a Python expression. See quotes around variables for instance.

use-lock

This option is used to ensure only one test is running at a time by using a file system lock. We use this to avoid having to deal with ports on different runtime service deps. (for instance using buildbot)

defaults

The defaults option lets you specify testrunner default options. These are specified as Python source for an expression yielding a list, typically a list literal.

environment

A set of environment variables that should be exported before starting the tests.

trunk (2010-08-24)

• xxx [Julien Anguenot]

0.3.3 (2010-08-24)

• fixed bug that was causing Buildout to pull in new (API-incompatible) version of Funkload; pinned version of Funkload [Raj Shah]

0.3.2 (2010-02-22)

• cleaned up code to use singleton (instead of decorator) to provide global entry point to get database session
• fixed bug that was causing new benches to populate database with old benches' results due to improper parsing of os.stat output [Raj Shah]
• fixed bug when computing start and stop runtime deps on Windows; Windows uses backslash as a path separator, and in the paths, eval() was trying to escape backslashes followed by characters (such as 'n', 't', etc.) [Raj Shah]
• fixed bug that didn't allow for updating conditions [Raj Shah]

0.3.1 (2010-02-03)

• fixed bug that didn't eval() DSN, causing DSN to be "stringified" an extra time [Raj Shah]

0.3 (2010-02-02)

• fixed syntax error typo [Raj Shah]
• fixed bug that didn't eval() DSN, causing DSN to be "stringified" an extra time [Raj Shah]

0.3 (2010-02-01)

• Fix version / release / tags [Julien Anguenot]

0.2.1 (2010-02-01)

• fixed bug that required dsn and passing-conditions to be declared which broke compatibility; made dsn and passing-conditions optional [Raj Shah]
• wrote performance builder code to automatically run Funkload test suite and populate database with results [Raj Shah]

O.2 (2009-10-31)

• Funkload conf and test case samples generation [Julien Anguenot]

O.2 (2009-10-28)

• xxx [Julien Anguenot]

Detailed Documentation