This spec describes the API for adding data to a Smoke database. It is
expected that Python, Perl, and command line interfaces will be generated
using this API. (Currently only a Python module and command line interface
exists.)
A Smoke database holds data about projects, machines, builds, regression
tests, and performance metrics. A build entry specifies the project to which
it belongs and the machine on which it was built. A regression test specifies
the build it is testing, the machine on which the test was run and the test
specification defining what kind of test it is. Etc. These relationships are
all reflected in the Client API methods described here.
As stated above a Smoke database stores information about projects, machines,
builds, tests, and performances metrics. There are methods to add each of
these data. Each such method returns a unique numeric id for the added
entry:
To add a build you need to know the machine id. To add a test you need to
know the build and machine ids. To that end there are methods to obtain the
relevant ids:
There are also several methods to add arbitrary name/value pairs (attributes)
to each of the data types. In the common case such attributes are not
necessary, so these methods may be safely ignored by most:
A number of this API's arguments are common to many of the methods.
For brevity, those arguments are described here. If applicable the associate
fallback environment variable is given in parentheses.
server (SMOKE_SERVER)
Communication with a Smoke database is via HTTP to a running Smoke
server. The 'server' argument is a full URL path to that Smoke Server.
project_id (SMOKE_PROJECTID)
is a unique numeric identifier for a project. (Usually this is an index
into the Smoke Server's backend database project table.)
project_name (SMOKE_PROJECTNAME)
is a unique name identifying a project. A Smoke server will enforce
uniqueness of project names.
machine_id (SMOKE_MACHINEID)
is a unique numeric identifier for a machine. (Usually this is an index
into the Smoke Server's backend database machine table.) Typically,
if machine_id cannot be determined it will fall back to the current
machine, adding a machine entry to the Smoke database if necessary.
build_id (SMOKE_BUILDID)
is a unique numeric identifier for a build. (Usually this is an index
into the Smoke Server's backend database build table.)
testspec_id (SMOKE_TESTSPECID)
is a unique numeric identifier for a regression test specification.
(Usually this is an index into the Smoke Server's backend database
testspec table.)
test_id (SMOKE_TESTID)
is a unique numeric identifier for a regression test. (Usually this is an index
into the Smoke Server's backend database test table.)
perfspec_id (SMOKE_PERF)
is a unique numeric identifier for a performance metric specification.
(Usually this is an index into the Smoke Server's backend database
perfspec table.)
perf_id (SMOKE_PERFID)
is a unique numeric identifier for a performance metric. (Usually this is
an index into the Smoke Server's backend database perf table.)
is a short unique name for the project. This name is used URLs when
browsing a Smoke server, so it is preferable to choose a name without
spaces or special charaters.
fullname (optional)
is a longer, more descriptive name for the project.
description (optional)
is a text description of the project.
attributes (optional)
is a hash of arbitrary name/value pairs to further describe the
project.
This adds a project entry and returns a unique numeric id for that project.
It is an error to add a project with the same name as an existing entry.
How an error is reported is implementation-dependent.
Note: It is up to the implementation whether to offer the "attributes"
argument. Some languages do not make this practical, and there is always the
addMachineAttribute() method.
Add (or update) arbitrary attributes for a given project.
Note: It is up to the implementation whether to offer the
updateProjectAttributes() method. Some languages do not make this practical,
and there is always the addProjectAttribute() method.
is the machine's name. If not specified it defaults to the name of
the current machine.
os_family
is a generic OS family name -- one of the following: Windows, Linux,
Mac, Solaris, HP-UX. If not specified it defaults to the OS family
of the current machine.
os_release
is the OS release. It enables differentiation of, say, Win95 and
Win98. (This may be redundant with "os" for some platforms.) If not
specified it defaults to the OS release of the current machine.
os
is a more specific name for the operating system: like 'win32',
'linux', 'solaris8' (XXX spec these). If not specified it defaults
to the OS of the current machine.
architecture
is one of the following archtecture names: 'ix86', 'sun4u' (XXX spec
these). If not specified it defaults to the architecture of the
current machine.
description (optional)
is an optional text description of the machine.
attributes (optional)
is a hash of arbitrary name/value pairs to further describe the
project.
This adds a machine entry and returns a unique numeric id for that machine.
It is an error to add a machine with the same (name, os_family, os_version,
os, architecture) combination. How an error is reported is
implementation-dependent.
Note: It is up to the implementation whether to offer the "attributes"
argument. Some languages do not make this practical, and there is always the
addMachineAttribute() method.
Returns the machine id for the current machine, OR if no such machine entry
exists, it creates a entry for the current machine and returns its new id.
Note: the behaviour of creating a table entry if it does not already exist is
unique the the getMachineId() method. This is NOT the behaviour for the other
get*Id() methods.
Add (or update) arbitrary attributes for a given machine.
Note: It is up to the implementation whether to offer the
updateMachineAttributes() method. Some languages do not make this practical,
and there is always the addMachineAttribute() method.
This adds an entry to the Smoke database's build table and returns the
database's unique id for that build.
The result, log_file and checksum fields are optional to allow
build entries for incomplete and failed builds. If is strongly recommended
that result be specified for completed builds and checksum be
specified for successful builds (both actions can be does with
updateBuild). If result is not specified, the Smoke server reporting
will be unable to know if a build has completed and a build's checksum is
the primary mechanism for determining a build id.
Note: It is up to the implementation whether to offer the "attributes"
argument. Some languages do not make this practical, and there is always the
addBuildAttribute() method.
Return a build id for development builds of the given package for this
project and version (on this machine). If one does not yet exist, it will be
created.
Build ids, as returned by addBuild and getBuildId are generally only
useful for complete builds generated by automated build machines. However,
it is sometimes useful to perform regression tests and generate performance
metrics on developement builds. If this data is to be added to a Smoke
database, these tests and perfs must refer to a build_id. The solution is to
allow special "development" builds that are static for a combination of
project_id, package, version and (optionally) build_flags.
For example, developer Joe is working on his project called Bling. The
current Bling version is 0.9.5 and the only relevant package for the Bling
project is the "installer package". To get a build_id for his development
builds, Joe would call:
Add (or update) arbitrary attributes for a given build.
Note: It is up to the implementation whether to offer the
updateBuildAttributes() method. Some languages do not make this practical,
and there is always the addBuildAttribute() method.
To add a test result you must first add a test specification entry and have
the added test refer to it. A test specification, or testspec, is uniquely
identifiable by its testspec_id or by its name and project_id
(using getTestSpecId).
addTestSpec(string name, [string fullname, string description,
int project_id, hash attributes, string server])
returns int testspec_id
where,
name
is a (preferable short) name identifying this test. A test
specification name must be unique for its project. This name may be
used in Smoke server URLs so it is recommended that spaces and
special characters be avoided.
fullname (optional)
is a longer, more descriptive name for the test specification.
Add (or update) arbitrary attributes for a given test specification.
Note: It is up to the implementation whether to offer the
updateTestSpecAttributes() method. Some languages do not make this practical,
and there is always the addTestSpecAttribute() method.
addTest(boolean result, [datetime starttime, file log_file,
int testspec_id, int build_id, int machine_id, hash attributes,
string server]) returns int test_id
where,
result
is a result for the test. True indicates that the test passed.
starttime
indicates when the test was started. If not specified this defaults
to the current time.
log_file (optional)
is a file containing a log of the test run. Some implementations of
this API may choose to also offer a "log" argument, which is the log
content itself.
This adds a test result of the given type (testspec_id) and returns a unique
id for that test.
Note: It is up to the implementation whether to offer the "attributes"
argument. Some languages do not make this practical, and there is always the
addTestAttribute() method.
Add (or update) arbitrary attributes for a given test.
Note: It is up to the implementation whether to offer the
updateTestAttributes() method. Some languages do not make this practical, and
there is always the addTestAttribute() method.
To add a performance measurement you must first add a performance metric
specification and have the added "perf" refer to it. A performance metric
specification, or perfspec, is uniquely identifiable by its perfspec_id
or by its name and project_id (using getPerfSpecId).
is a (preferable short) name identifying this performance metric. A
perf spec name must be unique for its project. This name may be used
in Smoke server URLs so it is recommended that spaces and special
characters be avoided.
fullname (optional)
is a longer, more descriptive name for the performance metric
specification.
description (optional)
is a description of the perf spec.
units (optional)
gives the units for the floating-point perf result and
target fields
target (optional)
is a target value for this metric. I.e. it is a development goal to
have this performance metric for this project reach this value. This
allows the Smoke server performance reporting to put measurements in
context.
more_is_good (optional)
is a boolean indicating if high results for this metric is a good
(e.g. number of pages served per minute) or bad (e.g. memory
footprint) thing.
Add (or update) arbitrary attributes for a given performance metric
specification.
Note: It is up to the implementation whether to offer the
updatePerfSpecAttributes() method. Some languages do not make this practical,
and there is always the addPerfSpecAttribute() method.
addPerf(float result, [datetime starttime, file log_file,
int perfspec_id, int build_id, int machine_id, hash attributes,
string server]) returns int perf_id
where,
result
is the measurement result (a floating-point value).
starttime
indicates when the measurement was started. If not specified this defaults
to the current time.
log_file (optional)
is a file containing a log of the performance run. Some
implementations of this API may choose to also offer a "log"
argument, which is the log content itself.
This adds a performance result of the given type (perfspec_id) and returns a unique
id for that performance result.
Note: It is up to the implementation whether to offer the "attributes"
argument. Some languages do not make this practical, and there is always the
addPerfAttribute() method.
Add (or update) arbitrary attributes for a given performance result.
Note: It is up to the implementation whether to offer the
updatePerfAttributes() method. Some languages do not make this practical, and
there is always the addPerfAttribute() method.
It may be useful to let a Smoke server know when a build has started and then
update that build entry's information when the build has completed. To do
this you can use the following pseudo-code:
# Indicate the start of the automated build.
project_id = smokeclient.getProjectId("Bling")
build_id = smokeclient.addBuild(package=<package-name>,
version=<version>,
project_id=<product>,
scc_revision=<scc_revision>)
# Build the product.
# ...
# After the build, indicate success or failure.
if success:
checksum = md5sum(<installer package content>)
smoke.updateBuild(build_id, result=sucess, log_file="build.log"
checksum=checksum)
else:
smoke.updateBuild(build_id, result=sucess, log_file="build.log")
