public class ThroughputMaster extends TaskMaster<ThroughputMaster.JobState,com.bigdata.service.jini.benchmark.ThroughputMaster.ClientTask,Void>
ISimpleSplitHandlerto ensure that separator keys are chosen for index partitions that fall on logical row boundaries.
Higher-level applications (such as the RDF DB) can introduce dependencies in the scale-out indices. For example, a write on the RDF DB must first write on TERM2ID, then on ID2TERM, and finally may write on the SPO, POS, and OSP indices in parallel.
You can examine the performance on the system at several levels:
AbstractFederationcollects performance counters from the underlying operating system using
vmstator the equivalent and relays them to the
LoadBalancerService. This is done (by default) for all machines participating in a federation, including those running clients.
AbstractFederationcollects statistics on its work queues, including the moving average of the queue length and the moving average of the queueing time. This information is collected for all clients (including the
DataServices) and is relayed to the
LoadBalancerService. This mechanism may be extended by an application to report additional performance counters. Clients MAY also report their own
DataServers collects statistics on its internal work queues, including the moving average of the various queue lengths and the moving average of the queueing time for those queues. The most interesting queue is generally the "write service", which is where tasks which will write on an index are queued. The
DataServeralso reports a variety of
Events to the
LoadBalancerService, most of which deal with synchronous and asynchronous overflow handling.
LoadBalancerServicenormally exposes an
CounterSetHTTPDinstance which may be used to examine the aggregated performance counters using a navigational model or as correlated timeseries and pivot tables suitable for import into a worksheet program. The
CounterSetHTTPDalso exposes an
XHTMLRenderer. In addition, any client or service MAY optionally expose a
CounterSetHTTPDinstance giving a "live" look at the performance counters for that host and service, but only the current value for each counter will be available (no timeseries). Finally, the
LoadBalancerServicelogs both the performance counters and the
Events. The performance counter logs are available for post-mortem analysis using
Events are logged in a tab-delimited format and may be imported into a worksheet for further analysis.
While bigdata does not report metrics via JMX, you can enable remote JMX and
examine JMX counters for JVMs throughout the federation. This can be done
trivially if you are running one
DataServer per host since you can
pre-assign the port number for remote JMX in the
|Modifier and Type||Class and Description|
State describing the job to be executed.
Run an unisolated operation.
|Modifier||Constructor and Description|
|Modifier and Type||Method and Description|
Extended to register a scale-out index in the specified namespace.
Runs the master.
Return a client to be executed on a remote data service.
allDone, attachPerformanceCounters, awaitAll, call, cancelAll, detachPerformanceCounters, error, execute, forceOverflow, getFederation, getJobState, innerMain, notifyOutcome, runJob, setupJob, startClients, success, tearDownJob
protected ThroughputMaster(JiniFederation fed) throws net.jini.config.ConfigurationException
public static void main(String args) throws net.jini.config.ConfigurationException, ExecutionException, InterruptedException, org.apache.zookeeper.KeeperException
Configurationand any overrides.
protected void beginJob(ThroughputMaster.JobState jobState) throws Exception
protected ThroughputMaster.JobState newJobState(String component, net.jini.config.Configuration config) throws net.jini.config.ConfigurationException
protected com.bigdata.service.jini.benchmark.ThroughputMaster.ClientTask newClientTask(int clientNum)
IBigdataFederationwhen it executes on the remote data service if it implements
IDataServiceCallable. You can use
AbstractClientTaskas a starting point.
Copyright © 2006-2014 SYSTAP, LLC. All Rights Reserved.