Package org.apache.storm.cluster
Interface IStormClusterState
-
- All Known Implementing Classes:
StormClusterStateImpl
public interface IStormClusterState
-
-
Method Summary
All Methods Instance Methods Abstract Methods Default Methods Deprecated Methods Modifier and Type Method Description voidactivateStorm(String stormId, StormBase stormBase, Map<String,Object> topoConf)List<String>activeKeys()List<String>activeStorms()voidaddNimbusHost(String nimbusId, NimbusSummary nimbusSummary)voidaddPrivateWorkerKey(WorkerTokenServiceType type, String topologyId, long keyVersion, PrivateWorkerKey key)Store a new version of a private key.default Map<String,SupervisorInfo>allSupervisorInfo()Get all of the supervisors with the ID as the key.default Map<String,SupervisorInfo>allSupervisorInfo(Runnable callback)Get all supervisor info.AssignmentassignmentInfo(String stormId, Runnable callback)Get the assignment based on storm id from local backend.VersionedData<Assignment>assignmentInfoWithVersion(String stormId, Runnable callback)List<String>assignments(Runnable callback)Map<String,Assignment>assignmentsInfo()Get all the topologies assignments mapping stormId -> Assignment from local backend.IntegerassignmentVersion(String stormId, Runnable callback)List<String>backpressureTopologies()Deprecated, for removal: This API element is subject to removal in a future version.List<String>blobstore(Runnable callback)List<String>blobstoreInfo(String blobKey)Credentialscredentials(String stormId, Runnable callback)voiddeleteTopologyProfileRequests(String stormId, ProfileRequest profileRequest)voiddisconnect()List<ErrorInfo>errors(String stormId, String componentId)List<String>errorTopologies()Map<ExecutorInfo,ExecutorBeat>executorBeats(String stormId, Map<List<Long>,NodeInfo> executorNodePort)NimbusInfogetLeader(Runnable callback)Get leader info from state store, which was written when a master gains leadership.longgetNextPrivateWorkerKeyVersion(WorkerTokenServiceType type, String topologyId)Get the next key version number that should be used for this topology id.PrivateWorkerKeygetPrivateWorkerKey(WorkerTokenServiceType type, String topologyId, long keyVersion)Get a private key used to validate a token is correct.default Optional<String>getTopoId(String topologyName)Get a topology ID from the name of a topology.List<ProfileRequest>getTopologyProfileRequests(String stormId)ClusterWorkerHeartbeatgetWorkerHeartbeat(String stormId, String node, Long port)List<ProfileRequest>getWorkerProfileRequests(String stormId, NodeInfo nodeInfo)List<String>heartbeatStorms()Set<String>idsOfTopologiesWithPrivateWorkerKeys()Get a list of all topologyIds that currently have private worker keys stored, of any kind.booleanisAssignmentsBackendSynchronized()Flag to indicate if the assignments synced successfully, seesyncRemoteAssignments(Map).booleanisPacemakerStateStore()Flag to indicate if the Pacameker is backend store.ErrorInfolastError(String stormId, String componentId)List<NimbusSummary>nimbuses()AssignmentremoteAssignmentInfo(String stormId, Runnable callback)Get the assignment based on storm id from remote state store, eg: ZK.voidremoveAllPrivateWorkerKeys(String topologyId)Remove all of the worker keys for a given topology.voidremoveBackpressure(String stormId)Deprecated, for removal: This API element is subject to removal in a future version.voidremoveBlobstoreKey(String blobKey)voidremoveExpiredPrivateWorkerKeys(String topologyId)Remove all keys for the given topology that have expired.voidremoveKeyVersion(String blobKey)voidremoveStorm(String stormId)voidremoveStormBase(String stormId)voidremoveWorkerBackpressure(String stormId, String node, Long port)Deprecated, for removal: This API element is subject to removal in a future version.voidremoveWorkerHeartbeat(String stormId, String node, Long port)voidreportError(String stormId, String componentId, String node, Long port, Throwable error)voidsetAssignment(String stormId, Assignment info, Map<String,Object> topoConf)voidsetAssignmentsBackendSynchronized()Mark the assignments as synced successfully, seeisAssignmentsBackendSynchronized().voidsetCredentials(String stormId, Credentials creds, Map<String,Object> topoConf)voidsetTopologyLogConfig(String stormId, LogConfig logConfig, Map<String,Object> topoConf)voidsetupBackpressure(String stormId, Map<String,Object> topoConf)Deprecated, for removal: This API element is subject to removal in a future version.voidsetupBlob(String key, NimbusInfo nimbusInfo, Integer versionInfo)voidsetupErrors(String stormId, Map<String,Object> topoConf)voidsetupHeatbeats(String stormId, Map<String,Object> topoConf)voidsetWorkerProfileRequest(String stormId, ProfileRequest profileRequest)StormBasestormBase(String stormId, Runnable callback)Get a storm base for a topology.StringstormId(String stormName)Get storm id from passed name, null if the name doesn't exist on cluster.voidsupervisorHeartbeat(String supervisorId, SupervisorInfo info)SupervisorInfosupervisorInfo(String supervisorId)List<String>supervisors(Runnable callback)voidsyncRemoteAssignments(Map<String,byte[]> remote)Sync the remote state store assignments to local backend, used when master gains leadership, seeorg.apache.storm.nimbus.LeaderListenerCallback.voidsyncRemoteIds(Map<String,String> ids)Sync all the active storm ids of the cluster, used now when master gains leadership.voidteardownHeartbeats(String stormId)voidteardownTopologyErrors(String stormId)booleantopologyBackpressure(String stormId, long timeoutMs, Runnable callback)Deprecated, for removal: This API element is subject to removal in a future version.default Map<String,StormBase>topologyBases()LogConfigtopologyLogConfig(String stormId, Runnable cb)voidupdateStorm(String stormId, StormBase newElems)voidworkerHeartbeat(String stormId, String node, Long port, ClusterWorkerHeartbeat info)
-
-
-
Method Detail
-
assignmentInfo
Assignment assignmentInfo(String stormId, Runnable callback)
Get the assignment based on storm id from local backend.- Parameters:
stormId- topology idcallback- callback function- Returns:
Assignment
-
remoteAssignmentInfo
Assignment remoteAssignmentInfo(String stormId, Runnable callback)
Get the assignment based on storm id from remote state store, eg: ZK.- Parameters:
stormId- topology idcallback- callback function- Returns:
Assignment
-
assignmentsInfo
Map<String,Assignment> assignmentsInfo()
Get all the topologies assignments mapping stormId -> Assignment from local backend.- Returns:
- stormId -> Assignment mapping
-
syncRemoteAssignments
void syncRemoteAssignments(Map<String,byte[]> remote)
Sync the remote state store assignments to local backend, used when master gains leadership, seeorg.apache.storm.nimbus.LeaderListenerCallback.- Parameters:
remote- assigned assignments for a specificIStormClusterStateinstance, usually a supervisor/node.
-
isAssignmentsBackendSynchronized
boolean isAssignmentsBackendSynchronized()
Flag to indicate if the assignments synced successfully, seesyncRemoteAssignments(Map).- Returns:
- true if is synced successfully
-
isPacemakerStateStore
boolean isPacemakerStateStore()
Flag to indicate if the Pacameker is backend store.- Returns:
- true if Pacemaker is being used as StateStore
-
setAssignmentsBackendSynchronized
void setAssignmentsBackendSynchronized()
Mark the assignments as synced successfully, seeisAssignmentsBackendSynchronized().
-
assignmentInfoWithVersion
VersionedData<Assignment> assignmentInfoWithVersion(String stormId, Runnable callback)
-
assignmentVersion
Integer assignmentVersion(String stormId, Runnable callback) throws Exception
- Throws:
Exception
-
nimbuses
List<NimbusSummary> nimbuses()
-
addNimbusHost
void addNimbusHost(String nimbusId, NimbusSummary nimbusSummary)
-
stormBase
StormBase stormBase(String stormId, Runnable callback)
Get a storm base for a topology.- Parameters:
stormId- the id of the topologycallback- something to call if the data changes (best effort)- Returns:
- the StormBase or null if it is not alive.
-
stormId
String stormId(String stormName)
Get storm id from passed name, null if the name doesn't exist on cluster.- Parameters:
stormName- storm name- Returns:
- storm id
-
syncRemoteIds
void syncRemoteIds(Map<String,String> ids)
Sync all the active storm ids of the cluster, used now when master gains leadership.- Parameters:
ids- stormName -> stormId mapping
-
getWorkerHeartbeat
ClusterWorkerHeartbeat getWorkerHeartbeat(String stormId, String node, Long port)
-
getWorkerProfileRequests
List<ProfileRequest> getWorkerProfileRequests(String stormId, NodeInfo nodeInfo)
-
getTopologyProfileRequests
List<ProfileRequest> getTopologyProfileRequests(String stormId)
-
setWorkerProfileRequest
void setWorkerProfileRequest(String stormId, ProfileRequest profileRequest)
-
deleteTopologyProfileRequests
void deleteTopologyProfileRequests(String stormId, ProfileRequest profileRequest)
-
executorBeats
Map<ExecutorInfo,ExecutorBeat> executorBeats(String stormId, Map<List<Long>,NodeInfo> executorNodePort)
-
supervisorInfo
SupervisorInfo supervisorInfo(String supervisorId)
-
teardownHeartbeats
void teardownHeartbeats(String stormId)
-
teardownTopologyErrors
void teardownTopologyErrors(String stormId)
-
backpressureTopologies
@Deprecated(forRemoval=true, since="2.0.0") List<String> backpressureTopologies()
Deprecated, for removal: This API element is subject to removal in a future version.Get backpressure topologies. Note: In Storm 2.0. Retained for enabling transition from 1.x. Will be removed soon.
-
getLeader
NimbusInfo getLeader(Runnable callback)
Get leader info from state store, which was written when a master gains leadership.Caution: it can not be used for fencing and is only for informational purposes because we use ZK as our backend now, which could have a overdue info of nodes.
- Parameters:
callback- callback func- Returns:
NimbusInfo
-
setTopologyLogConfig
void setTopologyLogConfig(String stormId, LogConfig logConfig, Map<String,Object> topoConf)
-
workerHeartbeat
void workerHeartbeat(String stormId, String node, Long port, ClusterWorkerHeartbeat info)
-
supervisorHeartbeat
void supervisorHeartbeat(String supervisorId, SupervisorInfo info)
-
topologyBackpressure
@Deprecated(forRemoval=true, since="2.0.0") boolean topologyBackpressure(String stormId, long timeoutMs, Runnable callback)
Deprecated, for removal: This API element is subject to removal in a future version.Get topoloy backpressure. Note: In Storm 2.0. Retained for enabling transition from 1.x. Will be removed soon.
-
setupBackpressure
@Deprecated(forRemoval=true, since="2.0.0") void setupBackpressure(String stormId, Map<String,Object> topoConf)
Deprecated, for removal: This API element is subject to removal in a future version.Setup backpressure. Note: In Storm 2.0. Retained for enabling transition from 1.x. Will be removed soon.
-
removeBackpressure
@Deprecated(forRemoval=true, since="2.0.0") void removeBackpressure(String stormId)
Deprecated, for removal: This API element is subject to removal in a future version.Remove backpressure. Note: In Storm 2.0. Retained for enabling transition from 1.x. Will be removed soon.
-
removeWorkerBackpressure
@Deprecated(forRemoval=true, since="2.0.0") void removeWorkerBackpressure(String stormId, String node, Long port)
Deprecated, for removal: This API element is subject to removal in a future version.Remove worker backpressure. Note: In Storm 2.0. Retained for enabling transition from 1.x. Will be removed soon.
-
activateStorm
void activateStorm(String stormId, StormBase stormBase, Map<String,Object> topoConf)
-
removeStormBase
void removeStormBase(String stormId)
-
setAssignment
void setAssignment(String stormId, Assignment info, Map<String,Object> topoConf)
-
setupBlob
void setupBlob(String key, NimbusInfo nimbusInfo, Integer versionInfo)
-
removeStorm
void removeStorm(String stormId)
-
removeBlobstoreKey
void removeBlobstoreKey(String blobKey)
-
removeKeyVersion
void removeKeyVersion(String blobKey)
-
reportError
void reportError(String stormId, String componentId, String node, Long port, Throwable error)
-
setCredentials
void setCredentials(String stormId, Credentials creds, Map<String,Object> topoConf)
-
credentials
Credentials credentials(String stormId, Runnable callback)
-
disconnect
void disconnect()
-
getPrivateWorkerKey
PrivateWorkerKey getPrivateWorkerKey(WorkerTokenServiceType type, String topologyId, long keyVersion)
Get a private key used to validate a token is correct. This is expected to be called from a privileged daemon, and the ACLs should be set up to only allow nimbus and these privileged daemons access to these private keys.- Parameters:
type- the type of service the key is for.topologyId- the topology id the key is for.keyVersion- the version of the key this is for.- Returns:
- the private key or null if it could not be found.
-
addPrivateWorkerKey
void addPrivateWorkerKey(WorkerTokenServiceType type, String topologyId, long keyVersion, PrivateWorkerKey key)
Store a new version of a private key. This is expected to only ever be called from nimbus. All ACLs however need to be setup to allow the given services access to the stored information.- Parameters:
type- the type of service this key is for.topologyId- the topology this key is forkeyVersion- the version of the key this is for.key- the key to store.
-
getNextPrivateWorkerKeyVersion
long getNextPrivateWorkerKeyVersion(WorkerTokenServiceType type, String topologyId)
Get the next key version number that should be used for this topology id. This is expected to only ever be called from nimbus, but it is acceptable if the ACLs are setup so that it can work from a privileged daemon for the given service.- Parameters:
type- the type of service this is for.topologyId- the topology id this is for.- Returns:
- the next version number. It should be 0 for a new topology id/service combination.
-
removeExpiredPrivateWorkerKeys
void removeExpiredPrivateWorkerKeys(String topologyId)
Remove all keys for the given topology that have expired. The number of keys should be small enough that doing an exhaustive scan of them all is acceptable as there is no guarantee that expiration time and version number are related. This should be for all service types. This is expected to only ever be called from nimbus and some ACLs may be setup so being called from other daemons will cause it to fail.- Parameters:
topologyId- the id of the topology to scan.
-
removeAllPrivateWorkerKeys
void removeAllPrivateWorkerKeys(String topologyId)
Remove all of the worker keys for a given topology. Used to clean up after a topology finishes. This is expected to only ever be called from nimbus and ideally should only ever work from nimbus.- Parameters:
topologyId- the topology to clean up after.
-
idsOfTopologiesWithPrivateWorkerKeys
Set<String> idsOfTopologiesWithPrivateWorkerKeys()
Get a list of all topologyIds that currently have private worker keys stored, of any kind. This is expected to only ever be called from nimbus.- Returns:
- the list of topology ids with any kind of private worker key stored.
-
allSupervisorInfo
default Map<String,SupervisorInfo> allSupervisorInfo()
Get all of the supervisors with the ID as the key.
-
allSupervisorInfo
default Map<String,SupervisorInfo> allSupervisorInfo(Runnable callback)
Get all supervisor info.- Parameters:
callback- be alerted if the list of supervisors change- Returns:
- All of the supervisors with the ID as the key
-
getTopoId
default Optional<String> getTopoId(String topologyName)
Get a topology ID from the name of a topology.- Parameters:
topologyName- the name of the topology to look for- Returns:
- the id of the topology or null if it is not alive.
-
-