com.bigdata.jini.start
Interface BigdataZooDefs

public interface BigdataZooDefs

Interface declaring constants that are used to name various znodes of interest. A DumpZookeeper sample output shows the layout of the znodes of interest within the federation.

 test-fed 
   locks 
     serviceConfigMonitor 
       com.bigdata.service.jini.DataServer 
         lock0000000000 (Ephemeral) 
       com.bigdata.service.jini.LoadBalancerServer 
         lock0000000000 (Ephemeral) 
       com.bigdata.service.jini.MetadataServer 
         lock0000000000 (Ephemeral) 
       com.bigdata.service.jini.TransactionServer 
         lock0000000000 (Ephemeral) 
     createPhysicalService 
   config 
     com.bigdata.service.jini.DataServer {DataServiceConfiguration}
       logicalService0000000001 
         election 
         physicalServiceea8d5902-c7b1-4dc2-9f01-dd343aed8230 (Ephemeral) {UUID}
       logicalService0000000000 
         election 
         physicalServiceb2bf8b98-da0c-42f5-ac65-027bf3304429 (Ephemeral) {UUID}
     com.bigdata.service.jini.LoadBalancerServer {LoadBalancerServiceConfiguration}
       logicalService0000000000 
         election 
         physicalService911a9b28-7396-4932-ab80-77078119e7e2 (Ephemeral) {UUID}
     com.bigdata.service.jini.MetadataServer {MetadataServiceConfiguration}
       logicalService0000000000 
         election 
         physicalServicec0f35d2e-0a20-40c4-bb76-c97e7cb72eb3 (Ephemeral) {UUID}
     com.bigdata.service.jini.TransactionServer {TransactionServiceConfiguration}
       logicalService0000000000 
         election 
         physicalService87522080-2da6-42be-84a8-4a863b420042 (Ephemeral) {UUID}
   services
       com.bigdata.service.jini.TransactionServer 
          instances (persistent znodes)
       com.bigdata.service.jini.LoadBalancerServer
           instances (persistent znodes)
       com.bigdata.service.jini.MetadataServer
          instances (persistent znodes) 
       com.bigdata.service.jini.DataServerServer
          instances (persistent znodes) 
 

Each ServiceConfiguration znode defines the service type, the target #of service instances, the replication count, etc for a service. The children of the configuration node are the logicalService instances and use CreateMode.PERSISTENT_SEQUENTIAL.

The children of a logicalService are "election" and "physicalServices". The election znode is a lock node that is used for master election and determining the failover chain for services which support failover. The children of the physicalServices CreateMode.EPHEMERAL znodes representing the physical service instances. Since they are ephemeral, zookeeper will remove them if the physical service dies.

Note: The bigdata services DO NOT use the SEQUENTIAL flag since they need to be able to restart with the save physical service znode. Instead, they create the physicalService znode using the ServiceID assigned by jini. Since the physical services are NOT sequentially generated, we maintain a ZLock for the logical service. Physical services contend for that lock and whichever one holds the lock is the primary. The order of the services in the lock queue is the failover order for the secondaries.

A ServiceConfigurationZNodeMonitorTask is run for the each discovered ServiceConfiguration znode and maintains watches on the dominated logical services znodes and physical service znodes. This allows it to observe changes in the target serviceCount for a given service type and the target replicationCount for a logical service of that type. If the #of logical services is under the target count, then we need to create a new logical service instance (just a znode) and set a watch on it (@todo anyone can create the new logical service since it only involves assigning a UUID, but we need an election or a lock to decide who actually does it so that we don't get a flood of logical services created. All watchers need to set a watch on the new logical service once it is created.) [note: we need the logical / physical distinction for services such as jini which are peers even before we introduce replication for bigdata services.] The ServicesManagerServer also sets a Watcher for each logical service of any service type. This allows it to observe the join and leave of physical service instances. If it observes that a logical service is under the target replication count (which either be read from the configuration node which is the parent of that logical service or must be copied onto the logical service when it is created) AND the host satisifies the IServiceConstraints, then it joins a priority election of ephemeral nodes (@todo where) and waits for up to a timeout for N peers to join. The winner of the election is the ServicesManagerServer on the host best suited to start the new service instance and it starts an instance of the service on the host where it is running. (@todo after the new service starts, the logical service node will gain a new child (in the last position). that will trigger the watch. if the logical service is still under the target, then the process will repeat.)

Version:

$Id$

Author:

Bryan Thompson

TODO:

Replicated bigdata services are created under a parent having an instance number assigned by zookeeper. The parent corresponds to a logical service. It is assigned a UUID for compatibility with the existing APIs (which do not really support replication). The children of the logical service node are the znodes for the physical instances of that logical service. Each physical service instance has a service UUID, which is assigned by jini sometime after it starts and then recorded in the zookeeper znode for that physical service instance. The physical service instances use an election to determine which of them is the primary, which are the secondaries, and the order for replicating data to the secondaries. The services create an EPHERMERAL znode when they (re)start. That znode contains the service UUID and is deleted automatically by zookeeper when the service's ZooKeeper client is closed. The services with persistent state DO NOT use the SEQUENTIAL flag since the same znode MUST be re-created if the service is restarted. Also, since the znodes are EPHEMERAL, no children are allowed. Therefore all behavior performed by the services occurs in queues, elections and other high-level data control structures using znodes elsewhere in zookeeper. FIXME javadoc edit.

Field Summary

static String	CONFIG The name of the child of the bigdata federation root zpath whose children are the ServiceConfiguration znodes.
static String	JOBS The zname of the child of the zroot where we put all the state for distributed jobs.
static String	LOCKS The zname of the child of the zroot where we put all of our lock nodes.
static String	LOCKS_CREATE_PHYSICAL_SERVICE Relative path to a child of the zroot that is a container for lock nodes.
static String	LOCKS_RESOURCES The relative zpath to a node whose children are ZLock acquired using the IResourceLockService.
static String	LOCKS_SERVICE_CONFIG_MONITOR The relative zpath to a node whose children are lock nodes.
static String	LOGICAL_SERVICE_PREFIX The prefix for the name of a znode that represents a logical service.
static String	MASTER_ELECTION The name of the znode that is a child of LOGICAL_SERVICE_PREFIX serving as a lock node for the election of the master physical service for that logical service.
static String	PHYSICAL_SERVICES_CONTAINER The name of the znodes whose children represent the physical service instances.
static String	ZSLASH The slash character used as zpath component delimiter.

Field Detail

ZSLASH

static final String ZSLASH

The slash character used as zpath component delimiter.

See Also:

Constant Field Values

CONFIG

static final String CONFIG

The name of the child of the bigdata federation root zpath whose children are the ServiceConfiguration znodes.

See Also:

Constant Field Values

LOCKS

static final String LOCKS

The zname of the child of the zroot where we put all of our lock nodes.

See Also:

Constant Field Values

JOBS

static final String JOBS

The zname of the child of the zroot where we put all the state for distributed jobs.

See Also:

Constant Field Values

LOCKS_CREATE_PHYSICAL_SERVICE

static final String LOCKS_CREATE_PHYSICAL_SERVICE

Relative path to a child of the zroot that is a container for lock nodes. There is at most one such lock node for each logical service instance of each service type. The lock nodes are named using the ServiceConfiguration.className followed by an underscore ("_") followed by the znode of the logical service instance. For example,

 com.bigdata.service.jini.DataServer_logicalService0000000000
 

Each such znode is a lock node. Its children represent processes contending for that lock. The lock node data is the zpath to the logicalService for which a new physicalService must be created. The ServiceConfiguration is fetched from that zpath and gives the IServiceConstraints that must be satisified.

The LOCKS_CREATE_PHYSICAL_SERVICE container is watched by the ServicesManagerService using a MonitorCreatePhysicalServiceLocksTask. Any time a new lock node is created in the container, the MonitorCreatePhysicalServiceLocksTask will contend for that lock. If it obtains the lock and can satisfy the IServiceConstraints for the new service, then it will attempt to create an instance of that service. If successful, it ZLock.destroyLock()s the lock node. Otherwise it releases the ZLock and sleeps a bit. Either it or another ServicesManagerServer will try again once they gain the ZLock.

See Also:

Constant Field Values

LOCKS_SERVICE_CONFIG_MONITOR

static final String LOCKS_SERVICE_CONFIG_MONITOR

The relative zpath to a node whose children are lock nodes. The children are named for the service type, e.g., TransactionServer. The task holding the ZLock is the one that handles state changes for ServiceConfiguration znode, including its children, which are the logical service instance znodes. Each ServicesManagerServer creates a ServiceConfigurationZNodeMonitorTask that contends for the locks which are the children of this znode.

See Also:

Constant Field Values

LOCKS_RESOURCES

static final String LOCKS_RESOURCES

The relative zpath to a node whose children are ZLock acquired using the IResourceLockService.

See Also:

Constant Field Values

LOGICAL_SERVICE_PREFIX

static final String LOGICAL_SERVICE_PREFIX

The prefix for the name of a znode that represents a logical service. This znode is a CreateMode.PERSISTENT_SEQUENTIAL child of the CONFIG znode. This znode should have two children: PHYSICAL_SERVICES_CONTAINER and MASTER_ELECTION.

See Also:

Constant Field Values

PHYSICAL_SERVICES_CONTAINER

static final String PHYSICAL_SERVICES_CONTAINER

The name of the znodes whose children represent the physical service instances.

Most bigdata services are persistent. Each service has a persistent identifier and persistent state. The persistent identifier is its ServiceID, which is assigned by jini. The persistent state of a bigdata service is stored in its service specific data directory. In order to support restart, persistent services are represented by PERSISTENT znodes. The data in the znode is the ServiceConfiguration used to (re-)start the service. When the service is shutdown, its ephemeral znodes (used for master election, etc.) will be destroyed, but its persistent znode and the ServiceConfiguration required for restart will remain.

Non-persistent services MUST use an EPHEMERAL znode which will be automatically deleted once zookeeper notices a timeout for that service.

For both persistent and non-persistent services, the znode of a physical service is named by its service UUID. Jini assigns a ServiceID when the service is first registered. Persistent services store their ServiceID in their data directory. The ServiceID for any jini service is available from ServiceItem.serviceID.

The bigdata APIs use a service UUID in order to decouple the core modules from jini. The ServiceID may be converted to a service UUID using JiniUtil.serviceID2UUID(ServiceID).

See Also:

Constant Field Values

MASTER_ELECTION

static final String MASTER_ELECTION

The name of the znode that is a child of LOGICAL_SERVICE_PREFIX serving as a lock node for the election of the master physical service for that logical service.

See Also:

ZLock, Constant Field Values