net.opentsdb.core

Class TSDB

java.lang.Object

net.opentsdb.core.TSDB

public final class TSDB
extends Object

Thread-safe implementation of the TSDB client.

This class is the central class of OpenTSDB. You use it to add new data points or query the database.

Constructor Summary

Constructors

Constructor and Description
TSDB(Config config) Constructor

Method Summary

Methods

Modifier and Type	Method and Description
com.stumbleupon.async.Deferred<Object>	addPoint(String metric, long timestamp, double value, Map<String,String> tags) Adds a double precision floating-point value data point in the TSDB.
com.stumbleupon.async.Deferred<Object>	addPoint(String metric, long timestamp, float value, Map<String,String> tags) Adds a single floating-point value data point in the TSDB.
com.stumbleupon.async.Deferred<Object>	addPoint(String metric, long timestamp, long value, Map<String,String> tags) Adds a single integer value data point in the TSDB.
byte[]	assignUid(String type, String name) Attempts to assign a UID to a name for the given type Used by the UniqueIdRpc call to generate IDs for new metrics, tagks or tagvs.
com.stumbleupon.async.Deferred<ArrayList<Object>>	checkNecessaryTablesExist() Verifies that the data and UID tables exist in HBase and optionally the tree and meta data tables if the user has enabled meta tracking or tree building
void	collectStats(StatsCollector collector) Collects the stats and metrics tracked by this instance.
byte[]	dataTable()
void	deleteAnnotation(Annotation note) Delete the annotation object from the search index
void	deleteTSMeta(String tsuid) Delete the timeseries meta object from the search index
void	deleteUIDMeta(UIDMeta meta) Delete the UID meta object from the search index
void	dropCaches() Discards all in-memory caches.
com.stumbleupon.async.Deferred<SearchQuery>	executeSearch(SearchQuery query) Executes a search query using the search plugin
com.stumbleupon.async.Deferred<Object>	flush() Forces a flush of any un-committed in memory data.
org.hbase.async.HBaseClient	getClient() Returns the configured HBase client
Config	getConfig() Getter that returns the configuration object
Histogram	getPutLatencyHistogram() Returns a latency histogram for Put RPCs used to store data points.
Histogram	getScanLatencyHistogram() Returns a latency histogram for Scan RPCs used to fetch data points.
byte[]	getUID(UniqueId.UniqueIdType type, String name) Attempts to find the UID matching a given name
com.stumbleupon.async.Deferred<String>	getUidName(UniqueId.UniqueIdType type, byte[] uid) Attempts to find the name for a unique identifier given a type
void	indexAnnotation(Annotation note) Index the given Annotation object via the configured search plugin
void	indexTSMeta(TSMeta meta) Index the given timeseries meta object via the configured search plugin
void	indexUIDMeta(UIDMeta meta) Index the given UID meta object via the configured search plugin
void	initializePlugins(boolean init_rpcs) Should be called immediately after construction to initialize plugins and objects that rely on such.
byte[]	metaTable()
static short	metrics_width()
WritableDataPoints	newDataPoints() Returns a new WritableDataPoints instance suitable for this TSDB.
Query	newQuery() Returns a new Query instance suitable for this TSDB.
com.stumbleupon.async.Deferred<Boolean>	processTSMetaThroughTrees(TSMeta meta) Processes the TSMeta through all of the trees if configured to do so
com.stumbleupon.async.Deferred<Object>	shutdown() Gracefully shuts down this TSD instance.
List<String>	suggestMetrics(String search) Given a prefix search, returns a few matching metric names.
List<String>	suggestMetrics(String search, int max_results) Given a prefix search, returns matching metric names.
List<String>	suggestTagNames(String search) Given a prefix search, returns a few matching tag names.
List<String>	suggestTagNames(String search, int max_results) Given a prefix search, returns matching tagk names.
List<String>	suggestTagValues(String search) Given a prefix search, returns a few matching tag values.
List<String>	suggestTagValues(String search, int max_results) Given a prefix search, returns matching tag values.
static short	tagk_width()
static short	tagv_width()
byte[]	treeTable()
int	uidCacheHits() Number of cache hits during lookups involving UIDs.
int	uidCacheMisses() Number of cache misses during lookups involving UIDs.
int	uidCacheSize() Number of cache entries currently in RAM for lookups involving UIDs.
byte[]	uidTable()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TSDB

public TSDB(Config config)

Constructor

Parameters:

config - An initialized configuration object

Since:

2.0

Method Detail

initializePlugins

public void initializePlugins(boolean init_rpcs)

Should be called immediately after construction to initialize plugins and objects that rely on such. It also moves most of the potential exception throwing code out of the constructor so TSDMain can shutdown clients and such properly.

Parameters:

init_rpcs - Whether or not to initialize RPC plugins as well

Throws:

RuntimeException - if the plugin path could not be processed

IllegalArgumentException - if a plugin could not be initialized

Since:

2.0

getClient

public final org.hbase.async.HBaseClient getClient()

Returns the configured HBase client

Returns:

The HBase client

Since:

2.0

getConfig

public final Config getConfig()

Getter that returns the configuration object

Returns:

The configuration object

Since:

2.0

getUidName

public com.stumbleupon.async.Deferred<String> getUidName(UniqueId.UniqueIdType type,
                                                byte[] uid)

Attempts to find the name for a unique identifier given a type

Parameters:

type - The type of UID

uid - The UID to search for

Returns:

The name of the UID object if found

Throws:

IllegalArgumentException - if the type is not valid

NoSuchUniqueId - if the UID was not found

Since:

2.0

getUID

public byte[] getUID(UniqueId.UniqueIdType type,
            String name)

Attempts to find the UID matching a given name

Parameters:

type - The type of UID

name - The name to search for

Throws:

IllegalArgumentException - if the type is not valid

NoSuchUniqueName - if the name was not found

Since:

2.0

checkNecessaryTablesExist

public com.stumbleupon.async.Deferred<ArrayList<Object>> checkNecessaryTablesExist()

Verifies that the data and UID tables exist in HBase and optionally the tree and meta data tables if the user has enabled meta tracking or tree building

Returns:

An ArrayList of objects to wait for

Throws:

TableNotFoundException

Since:

2.0

uidCacheHits

public int uidCacheHits()

Number of cache hits during lookups involving UIDs.

uidCacheMisses

public int uidCacheMisses()

Number of cache misses during lookups involving UIDs.

uidCacheSize

public int uidCacheSize()

Number of cache entries currently in RAM for lookups involving UIDs.

collectStats

public void collectStats(StatsCollector collector)

Collects the stats and metrics tracked by this instance.

Parameters:

collector - The collector to use.

getPutLatencyHistogram

public Histogram getPutLatencyHistogram()

Returns a latency histogram for Put RPCs used to store data points.

getScanLatencyHistogram

public Histogram getScanLatencyHistogram()

Returns a latency histogram for Scan RPCs used to fetch data points.

metrics_width

public static short metrics_width()

Returns:

the width, in bytes, of metric UIDs

tagk_width

public static short tagk_width()

Returns:

the width, in bytes, of tagk UIDs

tagv_width

public static short tagv_width()

Returns:

the width, in bytes, of tagv UIDs

newQuery

public Query newQuery()

Returns a new Query instance suitable for this TSDB.

newDataPoints

public WritableDataPoints newDataPoints()

Returns a new WritableDataPoints instance suitable for this TSDB.

If you want to add a single data-point, consider using addPoint(java.lang.String, long, long, java.util.Map) instead.

addPoint

public com.stumbleupon.async.Deferred<Object> addPoint(String metric,
                                              long timestamp,
                                              long value,
                                              Map<String,String> tags)

Adds a single integer value data point in the TSDB.

Parameters:

metric - A non-empty string.

timestamp - The timestamp associated with the value.

value - The value of the data point.

tags - The tags on this series. This map must be non-empty.

Returns:

A deferred object that indicates the completion of the request. The Object has not special meaning and can be null (think of it as Deferred<Void>). But you probably want to attach at least an errback to this Deferred to handle failures.

Throws:

IllegalArgumentException - if the timestamp is less than or equal to the previous timestamp added or 0 for the first timestamp, or if the difference with the previous timestamp is too large.

IllegalArgumentException - if the metric name is empty or contains illegal characters.

IllegalArgumentException - if the tags list is empty or one of the elements contains illegal characters.

org.hbase.async.HBaseException - (deferred) if there was a problem while persisting data.

addPoint

public com.stumbleupon.async.Deferred<Object> addPoint(String metric,
                                              long timestamp,
                                              double value,
                                              Map<String,String> tags)

Adds a double precision floating-point value data point in the TSDB.

Parameters:

metric - A non-empty string.

timestamp - The timestamp associated with the value.

value - The value of the data point.

tags - The tags on this series. This map must be non-empty.

Returns:

A deferred object that indicates the completion of the request. The Object has not special meaning and can be null (think of it as Deferred<Void>). But you probably want to attach at least an errback to this Deferred to handle failures.

Throws:

IllegalArgumentException - if the timestamp is less than or equal to the previous timestamp added or 0 for the first timestamp, or if the difference with the previous timestamp is too large.

IllegalArgumentException - if the metric name is empty or contains illegal characters.

IllegalArgumentException - if the value is NaN or infinite.

IllegalArgumentException - if the tags list is empty or one of the elements contains illegal characters.

org.hbase.async.HBaseException - (deferred) if there was a problem while persisting data.

Since:

1.2

addPoint

public com.stumbleupon.async.Deferred<Object> addPoint(String metric,
                                              long timestamp,
                                              float value,
                                              Map<String,String> tags)

Adds a single floating-point value data point in the TSDB.

Parameters:

metric - A non-empty string.

timestamp - The timestamp associated with the value.

value - The value of the data point.

tags - The tags on this series. This map must be non-empty.

Returns:

A deferred object that indicates the completion of the request. The Object has not special meaning and can be null (think of it as Deferred<Void>). But you probably want to attach at least an errback to this Deferred to handle failures.

Throws:

IllegalArgumentException - if the timestamp is less than or equal to the previous timestamp added or 0 for the first timestamp, or if the difference with the previous timestamp is too large.

IllegalArgumentException - if the metric name is empty or contains illegal characters.

IllegalArgumentException - if the value is NaN or infinite.

IllegalArgumentException - if the tags list is empty or one of the elements contains illegal characters.

org.hbase.async.HBaseException - (deferred) if there was a problem while persisting data.

flush

public com.stumbleupon.async.Deferred<Object> flush()
                                             throws org.hbase.async.HBaseException

Forces a flush of any un-committed in memory data.

For instance, any data point not persisted will be sent to HBase.

Returns:

A Deferred that will be called once all the un-committed data has been successfully and durably stored. The value of the deferred object return is meaningless and unspecified, and can be null.

Throws:

org.hbase.async.HBaseException - (deferred) if there was a problem sending un-committed data to HBase. Please refer to the HBaseException hierarchy to handle the possible failures. Some of them are easily recoverable by retrying, some are not.

shutdown

public com.stumbleupon.async.Deferred<Object> shutdown()

Gracefully shuts down this TSD instance.

The method must call shutdown() on all plugins as well as flush the compaction queue.

Returns:

A Deferred that will be called once all the un-committed data has been successfully and durably stored, and all resources used by this instance have been released. The value of the deferred object return is meaningless and unspecified, and can be null.

Throws:

org.hbase.async.HBaseException - (deferred) if there was a problem sending un-committed data to HBase. Please refer to the HBaseException hierarchy to handle the possible failures. Some of them are easily recoverable by retrying, some are not.

suggestMetrics

public List<String> suggestMetrics(String search)

Given a prefix search, returns a few matching metric names.

Parameters:

search - A prefix to search.

suggestMetrics

public List<String> suggestMetrics(String search,
                          int max_results)

Given a prefix search, returns matching metric names.

Parameters:

search - A prefix to search.

max_results - Maximum number of results to return.

Since:

2.0

suggestTagNames

public List<String> suggestTagNames(String search)

Given a prefix search, returns a few matching tag names.

Parameters:

search - A prefix to search.

suggestTagNames

public List<String> suggestTagNames(String search,
                           int max_results)

Given a prefix search, returns matching tagk names.

Parameters:

search - A prefix to search.

max_results - Maximum number of results to return.

Since:

2.0

suggestTagValues

public List<String> suggestTagValues(String search)

Given a prefix search, returns a few matching tag values.

Parameters:

search - A prefix to search.

suggestTagValues

public List<String> suggestTagValues(String search,
                            int max_results)

Given a prefix search, returns matching tag values.

Parameters:

search - A prefix to search.

max_results - Maximum number of results to return.

Since:

2.0

dropCaches

public void dropCaches()

Discards all in-memory caches.

Since:

1.1

assignUid

public byte[] assignUid(String type,
               String name)

Attempts to assign a UID to a name for the given type Used by the UniqueIdRpc call to generate IDs for new metrics, tagks or tagvs. The name must pass validation and if it's already assigned a UID, this method will throw an error with the proper UID. Otherwise if it can create the UID, it will be returned

Parameters:

type - The type of uid to assign, metric, tagk or tagv

name - The name of the uid object

Returns:

A byte array with the UID if the assignment was successful

Throws:

IllegalArgumentException - if the name is invalid or it already exists

Since:

2.0

uidTable

public byte[] uidTable()

Returns:

the name of the UID table as a byte array for client requests

dataTable

public byte[] dataTable()

Returns:

the name of the data table as a byte array for client requests

treeTable

public byte[] treeTable()

Returns:

the name of the tree table as a byte array for client requests

metaTable

public byte[] metaTable()

Returns:

the name of the meta table as a byte array for client requests

indexTSMeta

public void indexTSMeta(TSMeta meta)

Index the given timeseries meta object via the configured search plugin

Parameters:

meta - The meta data object to index

Since:

2.0

deleteTSMeta

public void deleteTSMeta(String tsuid)

Delete the timeseries meta object from the search index

Parameters:

tsuid - The TSUID to delete

Since:

2.0

indexUIDMeta

public void indexUIDMeta(UIDMeta meta)

Index the given UID meta object via the configured search plugin

Parameters:

meta - The meta data object to index

Since:

2.0

deleteUIDMeta

public void deleteUIDMeta(UIDMeta meta)

Delete the UID meta object from the search index

Parameters:

meta - The UID meta object to delete

Since:

2.0

indexAnnotation

public void indexAnnotation(Annotation note)

Index the given Annotation object via the configured search plugin

Parameters:

note - The annotation object to index

Since:

2.0

deleteAnnotation

public void deleteAnnotation(Annotation note)

Delete the annotation object from the search index

Parameters:

note - The annotation object to delete

Since:

2.0

processTSMetaThroughTrees

public com.stumbleupon.async.Deferred<Boolean> processTSMetaThroughTrees(TSMeta meta)

Processes the TSMeta through all of the trees if configured to do so

Parameters:

meta - The meta data to process

Since:

2.0

executeSearch

public com.stumbleupon.async.Deferred<SearchQuery> executeSearch(SearchQuery query)

Executes a search query using the search plugin

Parameters:

query - The query to execute

Returns:

A deferred object to wait on for the results to be fetched

Throws:

IllegalStateException - if the search plugin has not been enabled or configured

Since:

2.0