telemetry.benchmark | index telemetry/benchmark.py

`# Copyright 2014 The Chromium Authors. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.`

Modules
`` |  |

telemetry.internal.util.command_line telemetry.decorators | optparse telemetry.page.page_test | telemetry.internal.story_runner telemetry.web_perf.timeline_based_measurement |

Classes
`` |  |

__builtin__.object
  BenchmarkMetadata
exceptions.Exception(exceptions.BaseException)
  InvalidOptionsError
telemetry.internal.util.command_line.Command(telemetry.internal.util.command_line.ArgumentHandlerMixIn)
  Benchmark

class Benchmark(telemetry.internal.util.command_line.Command)
`` | `Base class for a Telemetry benchmark. A benchmark packages a measurement and a PageSet together. Benchmarks default to using TBM unless you override the value of Benchmark.test, or override the CreatePageTest method. New benchmarks should override CreateStorySet.
`

Method resolution order:
  Benchmark
  telemetry.internal.util.command_line.Command
  telemetry.internal.util.command_line.ArgumentHandlerMixIn
  __builtin__.object

Methods defined here:

CreatePageTest(self, options)
  `Return the PageTest for this Benchmark.
  Override this method for PageTest tests.
  Override, override CreateTimelineBasedMeasurementOptions to configure
  TimelineBasedMeasurement tests. Do not override both methods.
  Args:
  options: a browser_options.BrowserFinderOptions instance
  Returns:
  |test()| if |test| is a PageTest class.
  Otherwise, a TimelineBasedMeasurement instance.`

CreateStorySet(self, options)
  `Creates the instance of StorySet used to run the benchmark.
  Can be overridden by subclasses.`

CreateTimelineBasedMeasurementOptions(self)
  `Return the TimelineBasedMeasurementOptions for this Benchmark.
  Override this method to configure a TimelineBasedMeasurement benchmark.
  Otherwise, override CreatePageTest for PageTest tests. Do not override
  both methods.`

CustomizeBrowserOptions(self, options)
  `Add browser options that are required by this benchmark.`

GetMetadata(self)

GetTraceRerunCommands(self)

Run(self, finder_options)
  `Do not override this method.`

SetupBenchmarkDebugTraceRerunOptions(self, tbm_options)
  `Setup tracing categories associated with debug trace option.`

SetupBenchmarkDefaultTraceRerunOptions(self, tbm_options)
  `Setup tracing categories associated with default trace option.`

SetupTraceRerunOptions(self, browser_options, tbm_options)

__init__(self, max_failures=None)
  `Creates a new Benchmark.
  Args:
  max_failures: The number of story run's failures before bailing
  from executing subsequent page runs. If None, we never bail.`

Class methods defined here:

AddCommandLineArgs(cls, parser) from __builtin__.type

HasTraceRerunDebugOption(cls) from __builtin__.type

Name(cls) from __builtin__.type

ProcessCommandLineArgs(cls, parser, args) from __builtin__.type

SetArgumentDefaults(cls, parser) from __builtin__.type

ShouldDisable(cls, possible_browser) from __builtin__.type
  `Override this method to disable a benchmark under specific conditions.
  Supports logic too complex for simple Enabled and Disabled decorators.
  Decorators are still respected in cases where this function returns False.`

ValueCanBeAddedPredicate(cls, value, is_first_result) from __builtin__.type
  `Returns whether |value| can be added to the test results.
  Override this method to customize the logic of adding values to test
  results.
  Args:
  value: a value.Value instance (except failure.FailureValue,
  skip.SkipValue or trace.TraceValue which will always be added).
  is_first_result: True if |value| is the first result for its
  corresponding story.
  Returns:
  True if |value| should be added to the test results.
  Otherwise, it returns False.`

Data descriptors defined here:

max_failures

Data and other attributes defined here:

options = {}

test = <class 'telemetry.web_perf.timeline_based_measurement.TimelineBasedMeasurement'>
  `Collects multiple metrics based on their interaction records.
  A timeline based measurement shifts the burden of what metrics to collect onto
  the story under test. Instead of the measurement
  having a fixed set of values it collects, the story being tested
  issues (via javascript) an Interaction record into the user timing API that
  describing what is happening at that time, as well as a standardized set
  of flags describing the semantics of the work being done. The
  TimelineBasedMeasurement object collects a trace that includes both these
  interaction records, and a user-chosen amount of performance data using
  Telemetry's various timeline-producing APIs, tracing especially.
  It then passes the recorded timeline to different TimelineBasedMetrics based
  on those flags. As an example, this allows a single story run to produce
  load timing data, smoothness data, critical jank information and overall cpu
  usage information.
  For information on how to mark up a page to work with
  TimelineBasedMeasurement, refer to the
  perf.metrics.timeline_interaction_record module.
  Args:
  options: an instance of timeline_based_measurement.Options.
  results_wrapper: A class that has the __init__ method takes in
  the page_test_results object and the interaction record label. This
  class follows the ResultsWrapperInterface. Note: this class is not
  supported long term and to be removed when crbug.com/453109 is resolved.`

Class methods inherited from telemetry.internal.util.command_line.Command:

Description(cls) from __builtin__.type

main(cls, args=None) from __builtin__.type
  `Main method to run this command as a standalone script.`

Data descriptors inherited from telemetry.internal.util.command_line.ArgumentHandlerMixIn:

__dict__
  `dictionary for instance variables (if defined)`

__weakref__
  `list of weak references to the object (if defined)`

class BenchmarkMetadata(__builtin__.object)
`` |  | Methods defined here:

AsDict(self)

__init__(self, name, description='', rerun_options=None)

Data descriptors defined here:

__dict__
  `dictionary for instance variables (if defined)`

__weakref__
  `list of weak references to the object (if defined)`

description

name

rerun_options

class InvalidOptionsError(exceptions.Exception)
`` | `Raised for invalid benchmark options.`

Method resolution order:
  InvalidOptionsError
  exceptions.Exception
  exceptions.BaseException
  __builtin__.object

Data descriptors defined here:

__weakref__
  `list of weak references to the object (if defined)`

Methods inherited from exceptions.Exception:

__init__(...)
  `x.__init__(...) initializes x; see help(type(x)) for signature`

Data and other attributes inherited from exceptions.Exception:

__new__ = <built-in method __new__ of type object>
  `T.__new__(S, ...) -> a new object with type S, a subtype of T`

Methods inherited from exceptions.BaseException:

__delattr__(...)
  `x.__delattr__('name') <==> del x.name`

__getattribute__(...)
  `x.__getattribute__('name') <==> x.name`

__getitem__(...)
  `x.__getitem__(y) <==> x[y]`

__getslice__(...)
  `x.__getslice__(i, j) <==> x[i:j]
  Use of negative indices is not supported.`

__reduce__(...)

__repr__(...)
  `x.__repr__() <==> repr(x)`

__setattr__(...)
  `x.__setattr__('name', value) <==> x.name = value`

__setstate__(...)

__str__(...)
  `x.__str__() <==> str(x)`

__unicode__(...)

Data descriptors inherited from exceptions.BaseException:

__dict__

args

message

Functions
`` |  |

AddCommandLineArgs(parser)

ProcessCommandLineArgs(parser, args)