net.opentsdb.tree

Class Tree

java.lang.Object

net.opentsdb.tree.Tree

public final class Tree
extends Object

Represents a meta data tree in OpenTSDB that organizes timeseries into a hierarchical structure for navigation similar to a file system directory. Actual results are stored in Branch and Leaf objects while meta data about the tree is contained in this object.

A tree is built from a set of TreeRules. The rules are stored separately in the same row as the tree definition object, but can be loaded into the tree for processing and return from an RPC request. Building a tree consists of defining a tree, assigning one or more rules, and passing TSMeta objects through the rule set using a TreeBuilder. Results are then stored in separate rows as branch and leaf objects.

If TSMeta collides with something that has already been processed by a rule set, a collision will be recorded, via this object, in a separate column in a separate row for collisions. Likewise, if a tree is set to strict_match, TSMetas that fail to match the rule set will be recorded to a separate row. This class provides helper methods for fetching and storing these collisions and non-matched items.

Since:

2.0

Constructor Summary

Constructors

Constructor and Description
Tree() Default constructor necessary for de/serialization
Tree(int tree_id) Constructor that sets the tree ID and the created timestamp to the current time.
Tree(Tree original) Copy constructor that creates a completely independent copy of the original object.

Method Summary

Methods

Modifier and Type	Method and Description
void	addCollision(String tsuid, String existing_tsuid) Adds a TSUID to the collision local list, must then be synced with storage
void	addNotMatched(String tsuid, String message) Adds a TSUID to the not-matched local list when strict_matching is enabled.
void	addRule(TreeRule rule) Adds the given rule to the tree, replacing anything in the designated spot
static int	bytesToId(byte[] row_key) Attempts to convert the given byte array into an integer tree ID Note: You can give this method a full branch row key and it will only parse out the first TREE_ID_WIDTH bytes.
static byte[]	COLLISION_PREFIX()
boolean	copyChanges(Tree tree, boolean overwrite) Copies changes from the incoming tree into the local tree, overriding if called to.
com.stumbleupon.async.Deferred<Integer>	createNewTree(TSDB tsdb) Attempts to store the local tree in a new row, automatically assigning a new tree ID and returning the value.
static com.stumbleupon.async.Deferred<Boolean>	deleteTree(TSDB tsdb, int tree_id, boolean delete_definition) Attempts to delete all branches, leaves, collisions and not-matched entries for the given tree.
static com.stumbleupon.async.Deferred<List<Tree>>	fetchAllTrees(TSDB tsdb) Attempts to retrieve all trees from the UID table, including their rules.
static com.stumbleupon.async.Deferred<Map<String,String>>	fetchCollisions(TSDB tsdb, int tree_id, List<String> tsuids) Returns the collision set from storage for the given tree, optionally for only the list of TSUIDs provided.
static com.stumbleupon.async.Deferred<Map<String,String>>	fetchNotMatched(TSDB tsdb, int tree_id, List<String> tsuids) Returns the not-matched set from storage for the given tree, optionally for only the list of TSUIDs provided.
static com.stumbleupon.async.Deferred<Tree>	fetchTree(TSDB tsdb, int tree_id) Attempts to fetch the given tree from storage, loading the rule set at the same time.
com.stumbleupon.async.Deferred<Boolean>	flushCollisions(TSDB tsdb) Attempts to flush the collisions to storage.
com.stumbleupon.async.Deferred<Boolean>	flushNotMatched(TSDB tsdb) Attempts to flush the non-matches to storage.
Map<String,String>	getCollisions()
long	getCreated()
String	getDescription()
boolean	getEnabled()
String	getName()
String	getNotes()
Map<String,String>	getNotMatched()
TreeRule	getRule(int level, int order) Retrieves a single rule from the rule set given a level and order
Map<Integer,TreeMap<Integer,TreeRule>>	getRules()
boolean	getStoreFailures()
boolean	getStrictMatch()
int	getTreeId()
static byte[]	idToBytes(int tree_id) Converts the tree ID into a byte array TREE_ID_WIDTH in size
static byte[]	NOT_MATCHED_PREFIX()
void	setCreated(long created)
void	setDescription(String description)
void	setEnabled(boolean enabled)
void	setName(String name)
void	setNotes(String notes)
void	setStoreFailures(boolean store_failures)
void	setStrictMatch(boolean strict_match)
void	setTreeId(int treeId)
com.stumbleupon.async.Deferred<Boolean>	storeTree(TSDB tsdb, boolean overwrite) Attempts to store the tree definition via a CompareAndSet call.
String	toString()
static byte[]	TREE_FAMILY()
static int	TREE_ID_WIDTH()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Tree

public Tree()

Default constructor necessary for de/serialization

Tree

public Tree(int tree_id)

Constructor that sets the tree ID and the created timestamp to the current time.

Parameters:

tree_id - ID of this tree

Tree

public Tree(Tree original)

Copy constructor that creates a completely independent copy of the original object.

Parameters:

original - The original object to copy from

Throws:

PatternSyntaxException - if one of the rule's regex is invalid

Method Detail

toString

public String toString()

Overrides:

toString in class Object

Returns:

Information about the tree

copyChanges

public boolean copyChanges(Tree tree,
                  boolean overwrite)

Copies changes from the incoming tree into the local tree, overriding if called to. Only parses user mutable fields, excluding rules.

Parameters:

tree - The tree to copy from

overwrite - Whether or not to copy all values from the incoming tree

Returns:

True if there were changes, false if not

Throws:

IllegalArgumentException - if the incoming tree was invalid

addRule

public void addRule(TreeRule rule)

Adds the given rule to the tree, replacing anything in the designated spot

Parameters:

rule - The rule to add

Throws:

IllegalArgumentException - if the incoming rule was invalid

addCollision

public void addCollision(String tsuid,
                String existing_tsuid)

Adds a TSUID to the collision local list, must then be synced with storage

Parameters:

tsuid - TSUID to add to the set

Throws:

IllegalArgumentException - if the tsuid was invalid

addNotMatched

public void addNotMatched(String tsuid,
                 String message)

Adds a TSUID to the not-matched local list when strict_matching is enabled. Must be synced with storage.

Parameters:

tsuid - TSUID to add to the set

Throws:

IllegalArgumentException - if the tsuid was invalid

storeTree

public com.stumbleupon.async.Deferred<Boolean> storeTree(TSDB tsdb,
                                                boolean overwrite)

Attempts to store the tree definition via a CompareAndSet call.

Parameters:

tsdb - The TSDB to use for access

overwrite - Whether or not tree data should be overwritten

Returns:

True if the write was successful, false if an error occurred

Throws:

IllegalArgumentException - if the tree ID is missing or invalid

org.hbase.async.HBaseException - if a storage exception occurred

getRule

public TreeRule getRule(int level,
               int order)

Retrieves a single rule from the rule set given a level and order

Parameters:

level - The level where the rule resides

order - The order in the level where the rule resides

Returns:

The rule if found, null if not found

createNewTree

public com.stumbleupon.async.Deferred<Integer> createNewTree(TSDB tsdb)

Attempts to store the local tree in a new row, automatically assigning a new tree ID and returning the value. This method will scan the UID table for the maximum tree ID, increment it, store the new tree, and return the new ID. If no trees have been created, the returned ID will be "1". If we have reached the limit of trees for the system, as determined by TREE_ID_WIDTH, we will throw an exception.

Parameters:

tsdb - The TSDB to use for storage access

Returns:

A positive ID, greater than 0 if successful, 0 if there was an error

fetchTree

public static com.stumbleupon.async.Deferred<Tree> fetchTree(TSDB tsdb,
                                             int tree_id)

Attempts to fetch the given tree from storage, loading the rule set at the same time.

Parameters:

tsdb - The TSDB to use for access

tree_id - The Tree to fetch

Returns:

A tree object if found, null if the tree did not exist

Throws:

IllegalArgumentException - if the tree ID was invalid

org.hbase.async.HBaseException - if a storage exception occurred

JSONException - if the object could not be deserialized

fetchAllTrees

public static com.stumbleupon.async.Deferred<List<Tree>> fetchAllTrees(TSDB tsdb)

Attempts to retrieve all trees from the UID table, including their rules. If no trees were found, the result will be an empty list

Parameters:

tsdb - The TSDB to use for storage

Returns:

A list of tree objects. May be empty if none were found

fetchCollisions

public static com.stumbleupon.async.Deferred<Map<String,String>> fetchCollisions(TSDB tsdb,
                                                                 int tree_id,
                                                                 List<String> tsuids)

Returns the collision set from storage for the given tree, optionally for only the list of TSUIDs provided. Note: This can potentially be a large list if the rule set was written poorly and there were many timeseries so only call this without a list of TSUIDs if you feel confident the number is small.

Parameters:

tsdb - TSDB to use for storage access

tree_id - ID of the tree to fetch collisions for

tsuids - An optional list of TSUIDs to fetch collisions for. This may be empty or null, in which case all collisions for the tree will be returned.

Returns:

A list of collisions or null if nothing was found

Throws:

org.hbase.async.HBaseException - if there was an issue

IllegalArgumentException - if the tree ID was invalid

fetchNotMatched

public static com.stumbleupon.async.Deferred<Map<String,String>> fetchNotMatched(TSDB tsdb,
                                                                 int tree_id,
                                                                 List<String> tsuids)

Returns the not-matched set from storage for the given tree, optionally for only the list of TSUIDs provided. Note: This can potentially be a large list if the rule set was written poorly and there were many timeseries so only call this without a list of TSUIDs if you feel confident the number is small.

Parameters:

tsdb - TSDB to use for storage access

tree_id - ID of the tree to fetch non matches for

tsuids - An optional list of TSUIDs to fetch non-matches for. This may be empty or null, in which case all non-matches for the tree will be returned.

Returns:

A list of not-matched mappings or null if nothing was found

Throws:

org.hbase.async.HBaseException - if there was an issue

IllegalArgumentException - if the tree ID was invalid

deleteTree

public static com.stumbleupon.async.Deferred<Boolean> deleteTree(TSDB tsdb,
                                                 int tree_id,
                                                 boolean delete_definition)

Attempts to delete all branches, leaves, collisions and not-matched entries for the given tree. Optionally can delete the tree definition and rules as well. Warning: This call can take a long time to complete so it should only be done from a command line or issues once via RPC and allowed to process. Multiple deletes running at the same time on the same tree shouldn't be an issue but it's a waste of resources.

Parameters:

tsdb - The TSDB to use for storage access

tree_id - ID of the tree to delete

delete_definition - Whether or not the tree definition and rule set should be deleted as well

Returns:

True if the deletion completed successfully, false if there was an issue.

Throws:

org.hbase.async.HBaseException - if there was an issue

IllegalArgumentException - if the tree ID was invalid

idToBytes

public static byte[] idToBytes(int tree_id)

Converts the tree ID into a byte array TREE_ID_WIDTH in size

Parameters:

tree_id - The tree ID to convert

Returns:

The tree ID as a byte array

Throws:

IllegalArgumentException - if the Tree ID is invalid

bytesToId

public static int bytesToId(byte[] row_key)

Attempts to convert the given byte array into an integer tree ID Note: You can give this method a full branch row key and it will only parse out the first TREE_ID_WIDTH bytes.

Parameters:

row_key - The row key or tree ID as a byte array

Returns:

The tree ID as an integer value

Throws:

IllegalArgumentException - if the byte array is less than TREE_ID_WIDTH long

COLLISION_PREFIX

public static byte[] COLLISION_PREFIX()

Returns:

The configured collision column qualifier prefix

NOT_MATCHED_PREFIX

public static byte[] NOT_MATCHED_PREFIX()

Returns:

The configured not-matched column qualifier prefix

TREE_FAMILY

public static byte[] TREE_FAMILY()

Returns:

The family to use when storing tree data

flushCollisions

public com.stumbleupon.async.Deferred<Boolean> flushCollisions(TSDB tsdb)

Attempts to flush the collisions to storage. The storage call is a PUT so it will overwrite any existing columns, but since each column is the TSUID it should only exist once and the data shouldn't change. Note: This will also clear the local collisions map

Parameters:

tsdb - The TSDB to use for storage access

Returns:

A meaningless deferred (will always be true since we need to group it with tree store calls) for the caller to wait on

Throws:

org.hbase.async.HBaseException - if there was an issue

flushNotMatched

public com.stumbleupon.async.Deferred<Boolean> flushNotMatched(TSDB tsdb)

Attempts to flush the non-matches to storage. The storage call is a PUT so it will overwrite any existing columns, but since each column is the TSUID it should only exist once and the data shouldn't change. Note: This will also clear the local not_matched map

Parameters:

tsdb - The TSDB to use for storage access

Returns:

A meaningless deferred (will always be true since we need to group it with tree store calls) for the caller to wait on

Throws:

org.hbase.async.HBaseException - if there was an issue

TREE_ID_WIDTH

public static int TREE_ID_WIDTH()

Returns:

The width of the tree ID in bytes

getTreeId

public int getTreeId()

Returns:

The treeId

getName

public String getName()

Returns:

The name of the tree

getDescription

public String getDescription()

Returns:

An optional description of the tree

getNotes

public String getNotes()

Returns:

Optional notes about the tree

getStrictMatch

public boolean getStrictMatch()

Returns:

Whether or not strict matching is enabled

getEnabled

public boolean getEnabled()

Returns:

Whether or not the tree should process TSMeta objects

getStoreFailures

public boolean getStoreFailures()

Returns:

Whether or not to store not matched and collisions

getRules

public Map<Integer,TreeMap<Integer,TreeRule>> getRules()

Returns:

The tree's rule set

getNotMatched

public Map<String,String> getNotMatched()

Returns:

List of TSUIDs that did not match any rules

getCollisions

public Map<String,String> getCollisions()

Returns:

List of TSUIDs that were not stored due to collisions

getCreated

public long getCreated()

Returns:

When the tree was created, Unix epoch in seconds

setName

public void setName(String name)

Parameters:

name - A descriptive name for the tree

setDescription

public void setDescription(String description)

Parameters:

description - A brief description of the tree

setNotes

public void setNotes(String notes)

Parameters:

notes - Optional notes about the tree

setStrictMatch

public void setStrictMatch(boolean strict_match)

Parameters:

strict_match - Whether or not a TSUID must match all rules in the tree to be included

setEnabled

public void setEnabled(boolean enabled)

Parameters:

enabled - Whether or not this tree should process TSMeta objects

setStoreFailures

public void setStoreFailures(boolean store_failures)

Parameters:

store_failures - Whether or not to store not matched or collisions

setTreeId

public void setTreeId(int treeId)

Parameters:

treeId - ID of the tree, users cannot modify this

setCreated

public void setCreated(long created)

Parameters:

created - The time when this tree was created, Unix epoch in seconds