(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
(ns puppetlabs.http.client.metrics-test
|
|
|
|
|
(:require [clojure.test :refer :all]
|
|
|
|
|
[puppetlabs.http.client.async-unbuffered-test :as unbuffered-test]
|
|
|
|
|
[puppetlabs.trapperkeeper.services.webserver.jetty9-service :as jetty9]
|
|
|
|
|
[puppetlabs.trapperkeeper.testutils.bootstrap :as testutils]
|
|
|
|
|
[puppetlabs.trapperkeeper.testutils.logging :as testlogging]
|
|
|
|
|
[puppetlabs.trapperkeeper.testutils.webserver :as testwebserver]
|
|
|
|
|
[puppetlabs.http.client.async :as async]
|
|
|
|
|
[puppetlabs.http.client.sync :as sync]
|
|
|
|
|
[puppetlabs.http.client.common :as common]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
[puppetlabs.trapperkeeper.core :as tk]
|
2016-05-04 17:07:48 +00:00
|
|
|
[puppetlabs.http.client.metrics :as metrics]
|
|
|
|
|
[schema.test :as schema-test])
|
2016-05-04 22:13:40 +00:00
|
|
|
(:import (com.puppetlabs.http.client Async RequestOptions
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
ClientOptions ResponseBodyType Sync)
|
|
|
|
|
(com.codahale.metrics MetricRegistry)
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
(java.net SocketTimeoutException)
|
2016-05-04 22:13:40 +00:00
|
|
|
(java.util.concurrent TimeoutException)
|
|
|
|
|
(com.puppetlabs.http.client.metrics Metrics ClientTimer ClientMetricData)))
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
|
2016-05-04 17:07:48 +00:00
|
|
|
(use-fixtures :once schema-test/validate-schemas)
|
|
|
|
|
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(def metric-namespace "puppetlabs.http-client.experimental")
|
|
|
|
|
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
(tk/defservice test-metric-web-service
|
|
|
|
|
[[:WebserverService add-ring-handler]]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(init [this context]
|
|
|
|
|
(add-ring-handler
|
|
|
|
|
(fn [_] {:status 200 :body "Hello, World!"}) "/hello")
|
|
|
|
|
(add-ring-handler (fn [_]
|
|
|
|
|
(do
|
|
|
|
|
(Thread/sleep 5)
|
|
|
|
|
{:status 200 :body "short"}))
|
|
|
|
|
"/short")
|
|
|
|
|
(add-ring-handler (fn [_]
|
|
|
|
|
(do
|
|
|
|
|
(Thread/sleep 100)
|
|
|
|
|
{:status 200 :body "long"}))
|
|
|
|
|
"/long")
|
|
|
|
|
context))
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
|
|
|
|
|
(def hello-url "http://localhost:10000/hello")
|
|
|
|
|
(def short-url "http://localhost:10000/short")
|
|
|
|
|
(def long-url "http://localhost:10000/long")
|
|
|
|
|
|
2016-04-22 21:26:51 +00:00
|
|
|
(def short-name (format "%s.with-url.%s.full-response" metric-namespace short-url))
|
|
|
|
|
(def short-name-with-get (format "%s.with-url-and-method.%s.GET.full-response"
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
metric-namespace short-url))
|
2016-04-22 21:26:51 +00:00
|
|
|
(def short-name-with-post (format "%s.with-url-and-method.%s.POST.full-response"
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
metric-namespace short-url))
|
|
|
|
|
|
2016-04-22 21:26:51 +00:00
|
|
|
(def long-name (format "%s.with-url.%s.full-response" metric-namespace long-url))
|
|
|
|
|
(def long-name-with-method (format "%s.with-url-and-method.%s.GET.full-response"
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
metric-namespace long-url))
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(def long-foo-name
|
2016-04-22 21:26:51 +00:00
|
|
|
"puppetlabs.http-client.experimental.with-metric-id.foo.full-response")
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(def long-foo-bar-name
|
2016-04-22 21:26:51 +00:00
|
|
|
"puppetlabs.http-client.experimental.with-metric-id.foo.bar.full-response")
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(def long-foo-bar-baz-name
|
2016-04-22 21:26:51 +00:00
|
|
|
"puppetlabs.http-client.experimental.with-metric-id.foo.bar.baz.full-response")
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
|
2016-04-22 21:26:51 +00:00
|
|
|
(def hello-name (format "%s.with-url.%s.full-response" metric-namespace hello-url))
|
|
|
|
|
(def hello-name-with-method (format "%s.with-url-and-method.%s.GET.full-response"
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
metric-namespace hello-url))
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
|
|
|
|
|
(deftest metrics-test-java-async
|
|
|
|
|
(testing "metrics work with java async client"
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(testlogging/with-test-logging
|
|
|
|
|
(testutils/with-app-with-config
|
|
|
|
|
app
|
|
|
|
|
[jetty9/jetty9-service test-metric-web-service]
|
|
|
|
|
{:webserver {:port 10000}}
|
|
|
|
|
(let [metric-registry (MetricRegistry.)
|
|
|
|
|
hello-request-opts (RequestOptions. hello-url)
|
|
|
|
|
short-request-opts (RequestOptions. short-url)
|
|
|
|
|
long-request-opts (doto (RequestOptions. long-url)
|
|
|
|
|
(.setMetricId (into-array ["foo" "bar" "baz"])))]
|
|
|
|
|
(with-open [client (Async/createClient (doto (ClientOptions.)
|
|
|
|
|
(.setMetricRegistry metric-registry)))]
|
|
|
|
|
(-> client (.get hello-request-opts) (.deref)) ; warm it up
|
|
|
|
|
(let [short-response (-> client (.get short-request-opts) (.deref))
|
|
|
|
|
long-response (-> client (.get long-request-opts) (.deref))]
|
|
|
|
|
(-> client (.post short-request-opts) (.deref))
|
|
|
|
|
(is (= 200 (.getStatus short-response)))
|
|
|
|
|
(is (= "short" (slurp (.getBody short-response))))
|
|
|
|
|
(is (= 200 (.getStatus long-response)))
|
|
|
|
|
(is (= "long" (slurp (.getBody long-response))))
|
|
|
|
|
(.timer metric-registry "fake")
|
|
|
|
|
(let [client-metric-registry (.getMetricRegistry client)
|
|
|
|
|
client-metrics (Metrics/getClientMetrics client-metric-registry)
|
|
|
|
|
client-metrics-data (Metrics/getClientMetricsData client-metric-registry)
|
2016-05-05 11:48:07 +00:00
|
|
|
url-metrics (.getUrlTimers client-metrics)
|
|
|
|
|
url-and-method-metrics (.getUrlAndMethodTimers client-metrics)
|
|
|
|
|
metric-id-metrics (.getMetricIdTimers client-metrics)
|
|
|
|
|
url-metrics-data (.getUrlData client-metrics-data)
|
|
|
|
|
url-and-method-metrics-data (.getUrlAndMethodData client-metrics-data)
|
|
|
|
|
metric-id-metrics-data (.getMetricIdData client-metrics-data)
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
all-metrics (.getMetrics metric-registry)]
|
|
|
|
|
(testing ".getMetricRegistry returns the associated MetricRegistry"
|
|
|
|
|
(is (instance? MetricRegistry client-metric-registry)))
|
|
|
|
|
(testing "Metrics/getClientMetrics returns only http client metrics"
|
|
|
|
|
(is (= 11 (count all-metrics)))
|
|
|
|
|
(is (= 10 (+ (count url-metrics)
|
|
|
|
|
(count url-and-method-metrics)
|
|
|
|
|
(count metric-id-metrics))))
|
|
|
|
|
(is (= 10 (+ (count url-metrics-data)
|
|
|
|
|
(count url-and-method-metrics-data)
|
|
|
|
|
(count metric-id-metrics-data)))))
|
|
|
|
|
(testing ".getClientMetrics returns a map of category to array of timers"
|
|
|
|
|
(is (= (set (list hello-name short-name long-name))
|
|
|
|
|
(set (map #(.getMetricName %) url-metrics))
|
|
|
|
|
(set (map #(.getMetricName %) url-metrics-data))))
|
|
|
|
|
(is (= (set (list hello-name-with-method short-name-with-get
|
|
|
|
|
short-name-with-post long-name-with-method))
|
|
|
|
|
(set (map #(.getMetricName %) url-and-method-metrics))
|
|
|
|
|
(set (map #(.getMetricName %) url-and-method-metrics-data))))
|
|
|
|
|
(is (= (set (list long-foo-name long-foo-bar-name long-foo-bar-baz-name))
|
|
|
|
|
(set (map #(.getMetricName %) metric-id-metrics))
|
|
|
|
|
(set (map #(.getMetricName %) metric-id-metrics-data))))
|
|
|
|
|
(is (every? #(instance? ClientTimer %) url-metrics))
|
|
|
|
|
(is (every? #(instance? ClientTimer %) url-and-method-metrics))
|
|
|
|
|
(is (every? #(instance? ClientTimer %) metric-id-metrics)))
|
|
|
|
|
(testing ".getClientMetricsData returns a map of metric category to arrays of metric data"
|
|
|
|
|
(let [short-data (first (filter #(= short-name (.getMetricName %)) url-metrics-data))
|
|
|
|
|
short-data-get (first (filter #(= short-name-with-get (.getMetricName %))
|
|
|
|
|
url-and-method-metrics-data))
|
|
|
|
|
short-data-post (first (filter #(= short-name-with-post (.getMetricName %))
|
|
|
|
|
url-and-method-metrics-data))
|
|
|
|
|
long-data (first (filter #(= long-name (.getMetricName %)) url-metrics-data))]
|
|
|
|
|
(is (every? #(instance? ClientMetricData %)
|
|
|
|
|
(concat url-metrics-data
|
|
|
|
|
url-and-method-metrics-data
|
|
|
|
|
metric-id-metrics-data)))
|
|
|
|
|
|
|
|
|
|
(is (= short-name (.getMetricName short-data)))
|
|
|
|
|
(is (= 2 (.getCount short-data)))
|
|
|
|
|
(is (<= 5 (.getMean short-data)))
|
|
|
|
|
(is (<= 10 (.getAggregate short-data)))
|
|
|
|
|
|
|
|
|
|
(is (= short-name-with-get (.getMetricName short-data-get)))
|
|
|
|
|
(is (= 1 (.getCount short-data-get)))
|
|
|
|
|
(is (<= 5 (.getMean short-data-get)))
|
|
|
|
|
(is (<= 5 (.getAggregate short-data-get)))
|
|
|
|
|
|
|
|
|
|
(is (= short-name-with-post (.getMetricName short-data-post)))
|
|
|
|
|
(is (= 1 (.getCount short-data-post)))
|
|
|
|
|
(is (<= 5 (.getMean short-data-post)))
|
|
|
|
|
(is (<= 5 (.getAggregate short-data-post)))
|
|
|
|
|
|
|
|
|
|
(is (>= 1 (Math/abs (- (.getAggregate short-data)
|
|
|
|
|
(+ (.getAggregate short-data-get)
|
|
|
|
|
(.getAggregate short-data-post))))))
|
|
|
|
|
|
|
|
|
|
(is (= long-name (.getMetricName long-data)))
|
|
|
|
|
(is (= 1 (.getCount long-data)))
|
|
|
|
|
(is (<= 100 (.getMean long-data)))
|
|
|
|
|
(is (<= 100 (.getAggregate long-data)))
|
|
|
|
|
|
|
|
|
|
(is (> (.getAggregate long-data) (.getAggregate short-data))))))))
|
|
|
|
|
(with-open [client (Async/createClient (ClientOptions.))]
|
|
|
|
|
(testing ".getMetricRegistry returns nil if no metric registry passed in"
|
|
|
|
|
(is (= nil (.getMetricRegistry client))))))))))
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
|
|
|
|
|
(deftest metrics-test-clojure-async
|
|
|
|
|
(testing "metrics work with clojure async client"
|
|
|
|
|
(testlogging/with-test-logging
|
|
|
|
|
(testutils/with-app-with-config
|
|
|
|
|
app
|
|
|
|
|
[jetty9/jetty9-service test-metric-web-service]
|
|
|
|
|
{:webserver {:port 10000}}
|
|
|
|
|
(let [metric-registry (MetricRegistry.)]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(with-open [client (async/create-client
|
|
|
|
|
{:metric-registry metric-registry})]
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
@(common/get client hello-url) ; warm it up
|
|
|
|
|
(let [short-response @(common/get client short-url {:as :text :metric-id ["foo" "bar" "baz"]})
|
|
|
|
|
long-response @(common/get client long-url)]
|
|
|
|
|
@(common/post client short-url)
|
|
|
|
|
(is (= {:status 200 :body "short"} (select-keys short-response [:status :body])))
|
|
|
|
|
(is (= 200 (:status long-response)))
|
|
|
|
|
(is (= "long" (slurp (:body long-response))))
|
|
|
|
|
(.timer metric-registry "fake")
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(let [client-metric-registry (common/get-client-metric-registry client)
|
|
|
|
|
client-metrics (metrics/get-client-metrics client-metric-registry)
|
|
|
|
|
client-metrics-data (metrics/get-client-metrics-data client-metric-registry)
|
|
|
|
|
url-metrics (:url client-metrics)
|
|
|
|
|
url-and-method-metrics (:url-and-method client-metrics)
|
|
|
|
|
metric-id-metrics (:metric-id client-metrics)
|
|
|
|
|
url-metrics-data (:url client-metrics-data)
|
|
|
|
|
url-and-method-metrics-data (:url-and-method client-metrics-data)
|
|
|
|
|
metric-id-metrics-data (:metric-id client-metrics-data)
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
all-metrics (.getMetrics metric-registry)]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(testing "get-client-metric-registry returns the associated MetricRegistry"
|
|
|
|
|
(is (instance? MetricRegistry client-metric-registry)))
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
(testing "get-client-metrics and get-client-metrics data return only http client metrics"
|
|
|
|
|
(is (= 11 (count all-metrics)))
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(is (= 10 (+ (count url-metrics)
|
|
|
|
|
(count url-and-method-metrics)
|
|
|
|
|
(count metric-id-metrics))))
|
|
|
|
|
(is (= 10 (+ (count url-metrics-data)
|
|
|
|
|
(count url-and-method-metrics-data)
|
|
|
|
|
(count metric-id-metrics-data)))))
|
|
|
|
|
(testing "get-client-metrics returns a map of category to array of timers"
|
|
|
|
|
(is (= (set (list hello-name short-name long-name))
|
|
|
|
|
(set (map #(.getMetricName %) url-metrics))
|
|
|
|
|
(set (map :metric-name url-metrics-data))))
|
|
|
|
|
(is (= (set (list hello-name-with-method short-name-with-get
|
|
|
|
|
short-name-with-post long-name-with-method))
|
|
|
|
|
(set (map #(.getMetricName %) url-and-method-metrics))
|
|
|
|
|
(set (map :metric-name url-and-method-metrics-data))))
|
|
|
|
|
(is (= (set (list long-foo-name long-foo-bar-name long-foo-bar-baz-name))
|
|
|
|
|
(set (map #(.getMetricName %) metric-id-metrics))
|
|
|
|
|
(set (map :metric-name metric-id-metrics-data))))
|
|
|
|
|
(is (every? #(instance? ClientTimer %) url-metrics))
|
|
|
|
|
(is (every? #(instance? ClientTimer %) url-and-method-metrics))
|
|
|
|
|
(is (every? #(instance? ClientTimer %) metric-id-metrics)))
|
|
|
|
|
(testing "get-client-metrics-data returns a map of metric category to metric data"
|
|
|
|
|
(let [short-data (first (filter #(= short-name (:metric-name %)) url-metrics-data))
|
|
|
|
|
short-data-get (first (filter #(= short-name-with-get (:metric-name %))
|
|
|
|
|
url-and-method-metrics-data))
|
|
|
|
|
short-data-post (first (filter #(= short-name-with-post (:metric-name %))
|
|
|
|
|
url-and-method-metrics-data))
|
|
|
|
|
long-data (first (filter #(= long-name (:metric-name %)) url-metrics-data))]
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
(is (= short-name (:metric-name short-data)))
|
|
|
|
|
(is (= 2 (:count short-data)))
|
|
|
|
|
(is (<= 5 (:mean short-data)))
|
|
|
|
|
(is (<= 10 (:aggregate short-data)))
|
|
|
|
|
|
|
|
|
|
(is (= short-name-with-get (:metric-name short-data-get)))
|
|
|
|
|
(is (= 1 (:count short-data-get)))
|
|
|
|
|
(is (<= 5 (:mean short-data-get)))
|
|
|
|
|
(is (<= 5 (:aggregate short-data-get)))
|
|
|
|
|
|
|
|
|
|
(is (= short-name-with-post (:metric-name short-data-post)))
|
|
|
|
|
(is (= 1 (:count short-data-post)))
|
|
|
|
|
(is (<= 5 (:mean short-data-post)))
|
|
|
|
|
(is (<= 5 (:aggregate short-data-post)))
|
|
|
|
|
|
|
|
|
|
(is (>= 1 (Math/abs (- (:aggregate short-data)
|
|
|
|
|
(+ (:aggregate short-data-get)
|
|
|
|
|
(:aggregate short-data-post))))))
|
|
|
|
|
|
|
|
|
|
(is (= long-name (:metric-name long-data)))
|
|
|
|
|
(is (= 1 (:count long-data)))
|
|
|
|
|
(is (<= 100 (:mean long-data)))
|
|
|
|
|
(is (<= 100 (:aggregate long-data)))
|
|
|
|
|
|
|
|
|
|
(is (> (:mean long-data) (:mean short-data)))))))))
|
|
|
|
|
(with-open [client (async/create-client {})]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(testing "get-client-metric-registry returns nil if no metric registry passed in"
|
|
|
|
|
(is (= nil (common/get-client-metric-registry client)))))))))
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
|
|
|
|
|
(deftest metrics-test-java-sync
|
|
|
|
|
(testing "metrics work with java sync client"
|
|
|
|
|
(testlogging/with-test-logging
|
|
|
|
|
(testutils/with-app-with-config
|
|
|
|
|
app
|
|
|
|
|
[jetty9/jetty9-service test-metric-web-service]
|
|
|
|
|
{:webserver {:port 10000}}
|
|
|
|
|
(let [metric-registry (MetricRegistry.)
|
|
|
|
|
hello-request-opts (RequestOptions. hello-url)
|
|
|
|
|
short-request-opts (RequestOptions. short-url)
|
|
|
|
|
long-request-opts (doto (RequestOptions. long-url)
|
|
|
|
|
(.setMetricId (into-array ["foo" "bar" "baz"])))]
|
|
|
|
|
(with-open [client (Sync/createClient (doto (ClientOptions.)
|
|
|
|
|
(.setMetricRegistry metric-registry)))]
|
|
|
|
|
(.get client hello-request-opts) ; warm it up
|
|
|
|
|
(let [short-response (.get client short-request-opts)
|
|
|
|
|
long-response (.get client long-request-opts)]
|
|
|
|
|
(.post client short-request-opts)
|
|
|
|
|
(is (= 200 (.getStatus short-response)))
|
|
|
|
|
(is (= "short" (slurp (.getBody short-response))))
|
|
|
|
|
(is (= 200 (.getStatus long-response)))
|
|
|
|
|
(is (= "long" (slurp (.getBody long-response))))
|
|
|
|
|
(.timer metric-registry "fake")
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(let [client-metric-registry (.getMetricRegistry client)
|
|
|
|
|
client-metrics (Metrics/getClientMetrics client-metric-registry)
|
|
|
|
|
client-metrics-data (Metrics/getClientMetricsData client-metric-registry)
|
2016-05-05 11:48:07 +00:00
|
|
|
url-metrics (.getUrlTimers client-metrics)
|
|
|
|
|
url-and-method-metrics (.getUrlAndMethodTimers client-metrics)
|
|
|
|
|
metric-id-metrics (.getMetricIdTimers client-metrics)
|
|
|
|
|
url-metrics-data (.getUrlData client-metrics-data)
|
|
|
|
|
url-and-method-metrics-data (.getUrlAndMethodData client-metrics-data)
|
|
|
|
|
metric-id-metrics-data (.getMetricIdData client-metrics-data)
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
all-metrics (.getMetrics metric-registry)]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(testing ".getMetricRegistry returns the associated MetricRegistry"
|
|
|
|
|
(is (instance? MetricRegistry client-metric-registry)))
|
|
|
|
|
(testing "Metrics/getClientMetrics returns only http client metrics"
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
(is (= 11 (count all-metrics)))
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(is (= 10 (+ (count url-metrics)
|
|
|
|
|
(count url-and-method-metrics)
|
|
|
|
|
(count metric-id-metrics))))
|
|
|
|
|
(is (= 10 (+ (count url-metrics-data)
|
|
|
|
|
(count url-and-method-metrics-data)
|
|
|
|
|
(count metric-id-metrics-data)))))
|
|
|
|
|
(testing ".getClientMetrics returns a map of category to array of timers"
|
|
|
|
|
|
|
|
|
|
(is (= (set (list hello-name short-name long-name))
|
|
|
|
|
(set (map #(.getMetricName %) url-metrics))
|
|
|
|
|
(set (map #(.getMetricName %) url-metrics-data))))
|
|
|
|
|
(is (= (set (list hello-name-with-method short-name-with-get
|
|
|
|
|
short-name-with-post long-name-with-method))
|
|
|
|
|
(set (map #(.getMetricName %) url-and-method-metrics))
|
|
|
|
|
(set (map #(.getMetricName %) url-and-method-metrics-data))))
|
|
|
|
|
(is (= (set (list long-foo-name long-foo-bar-name long-foo-bar-baz-name))
|
|
|
|
|
(set (map #(.getMetricName %) metric-id-metrics))
|
|
|
|
|
(set (map #(.getMetricName %) metric-id-metrics-data))))
|
|
|
|
|
(is (every? #(instance? ClientTimer %) url-metrics))
|
|
|
|
|
(is (every? #(instance? ClientTimer %) url-and-method-metrics))
|
|
|
|
|
(is (every? #(instance? ClientTimer %) metric-id-metrics)))
|
|
|
|
|
(testing ".getClientMetricsData returns a map of metric category to arrays of metric data"
|
|
|
|
|
(let [short-data (first (filter #(= short-name (.getMetricName %)) url-metrics-data))
|
|
|
|
|
short-data-get (first (filter #(= short-name-with-get (.getMetricName %))
|
|
|
|
|
url-and-method-metrics-data))
|
|
|
|
|
short-data-post (first (filter #(= short-name-with-post (.getMetricName %))
|
|
|
|
|
url-and-method-metrics-data))
|
|
|
|
|
long-data (first (filter #(= long-name (.getMetricName %)) url-metrics-data))]
|
|
|
|
|
(is (every? #(instance? ClientMetricData %)
|
|
|
|
|
(concat url-metrics-data
|
|
|
|
|
url-and-method-metrics-data
|
|
|
|
|
metric-id-metrics-data)))
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
|
|
|
|
|
(is (= short-name (.getMetricName short-data)))
|
|
|
|
|
(is (= 2 (.getCount short-data)))
|
|
|
|
|
(is (<= 5 (.getMean short-data)))
|
|
|
|
|
(is (<= 10 (.getAggregate short-data)))
|
|
|
|
|
|
|
|
|
|
(is (= short-name-with-get (.getMetricName short-data-get)))
|
|
|
|
|
(is (= 1 (.getCount short-data-get)))
|
|
|
|
|
(is (<= 5 (.getMean short-data-get)))
|
|
|
|
|
(is (<= 5 (.getAggregate short-data-get)))
|
|
|
|
|
|
|
|
|
|
(is (= short-name-with-post (.getMetricName short-data-post)))
|
|
|
|
|
(is (= 1 (.getCount short-data-post)))
|
|
|
|
|
(is (<= 5 (.getMean short-data-post)))
|
|
|
|
|
(is (<= 5 (.getAggregate short-data-post)))
|
|
|
|
|
|
|
|
|
|
(is (>= 1 (Math/abs (- (.getAggregate short-data)
|
|
|
|
|
(+ (.getAggregate short-data-get)
|
|
|
|
|
(.getAggregate short-data-post))))))
|
|
|
|
|
|
|
|
|
|
(is (= long-name (.getMetricName long-data)))
|
|
|
|
|
(is (= 1 (.getCount long-data)))
|
|
|
|
|
(is (<= 100 (.getMean long-data)))
|
|
|
|
|
(is (<= 100 (.getAggregate long-data)))
|
|
|
|
|
|
|
|
|
|
(is (> (.getMean long-data) (.getMean short-data))))))))
|
|
|
|
|
(with-open [client (Sync/createClient (ClientOptions.))]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(testing ".getMetricRegistry returns nil if no metric registry passed in"
|
|
|
|
|
(is (= nil (.getMetricRegistry client))))))))))
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
|
|
|
|
|
(deftest metrics-test-clojure-sync
|
|
|
|
|
(testing "metrics work with clojure sync client"
|
|
|
|
|
(testlogging/with-test-logging
|
|
|
|
|
(testutils/with-app-with-config
|
|
|
|
|
app
|
|
|
|
|
[jetty9/jetty9-service test-metric-web-service]
|
|
|
|
|
{:webserver {:port 10000}}
|
|
|
|
|
(let [metric-registry (MetricRegistry.)]
|
|
|
|
|
(with-open [client (sync/create-client {:metric-registry metric-registry})]
|
|
|
|
|
(common/get client hello-url) ; warm it up
|
|
|
|
|
(let [short-response (common/get client short-url {:as :text})
|
|
|
|
|
long-response (common/get client long-url {:as :text :metric-id ["foo" "bar" "baz"]})]
|
|
|
|
|
(common/post client short-url)
|
|
|
|
|
(is (= {:status 200 :body "short"} (select-keys short-response [:status :body])))
|
|
|
|
|
(is (= {:status 200 :body "long"} (select-keys long-response [:status :body])))
|
|
|
|
|
(.timer metric-registry "fake")
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(let [client-metric-registry (common/get-client-metric-registry client)
|
|
|
|
|
client-metrics (metrics/get-client-metrics client-metric-registry)
|
|
|
|
|
client-metrics-data (metrics/get-client-metrics-data client-metric-registry)
|
|
|
|
|
url-metrics (:url client-metrics)
|
|
|
|
|
url-and-method-metrics (:url-and-method client-metrics)
|
|
|
|
|
metric-id-metrics (:metric-id client-metrics)
|
|
|
|
|
url-metrics-data (:url client-metrics-data)
|
|
|
|
|
url-and-method-metrics-data (:url-and-method client-metrics-data)
|
|
|
|
|
metric-id-metrics-data (:metric-id client-metrics-data)
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
all-metrics (.getMetrics metric-registry)]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(testing "get-client-metric-registry returns the associated MetricRegistry"
|
|
|
|
|
(is (instance? MetricRegistry client-metric-registry)))
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
(testing "get-client-metrics and get-client-metrics data return only http client metrics"
|
|
|
|
|
(is (= 11 (count all-metrics)))
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(is (= 10 (+ (count url-metrics)
|
|
|
|
|
(count url-and-method-metrics)
|
|
|
|
|
(count metric-id-metrics))))
|
|
|
|
|
(is (= 10 (+ (count url-metrics-data)
|
|
|
|
|
(count url-and-method-metrics-data)
|
|
|
|
|
(count metric-id-metrics-data)))))
|
|
|
|
|
(testing "get-client-metrics returns a map of category to array of timers"
|
|
|
|
|
(is (= (set (list hello-name short-name long-name))
|
|
|
|
|
(set (map #(.getMetricName %) url-metrics))
|
|
|
|
|
(set (map :metric-name url-metrics-data))))
|
|
|
|
|
(is (= (set (list hello-name-with-method short-name-with-get
|
|
|
|
|
short-name-with-post long-name-with-method))
|
|
|
|
|
(set (map #(.getMetricName %) url-and-method-metrics))
|
|
|
|
|
(set (map :metric-name url-and-method-metrics-data))))
|
|
|
|
|
(is (= (set (list long-foo-name long-foo-bar-name long-foo-bar-baz-name))
|
|
|
|
|
(set (map #(.getMetricName %) metric-id-metrics))
|
|
|
|
|
(set (map :metric-name metric-id-metrics-data))))
|
|
|
|
|
(is (every? #(instance? ClientTimer %) url-metrics))
|
|
|
|
|
(is (every? #(instance? ClientTimer %) url-and-method-metrics))
|
|
|
|
|
(is (every? #(instance? ClientTimer %) metric-id-metrics)))
|
|
|
|
|
(testing "get-client-metrics-data returns a map of metric category to metric data"
|
|
|
|
|
(let [short-data (first (filter #(= short-name (:metric-name %)) url-metrics-data))
|
|
|
|
|
short-data-get (first (filter #(= short-name-with-get (:metric-name %))
|
|
|
|
|
url-and-method-metrics-data))
|
|
|
|
|
short-data-post (first (filter #(= short-name-with-post (:metric-name %))
|
|
|
|
|
url-and-method-metrics-data))
|
|
|
|
|
long-data (first (filter #(= long-name (:metric-name %)) url-metrics-data))]
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
(is (= short-name (:metric-name short-data)))
|
|
|
|
|
(is (= 2 (:count short-data)))
|
|
|
|
|
(is (<= 5 (:mean short-data)))
|
|
|
|
|
(is (<= 10 (:aggregate short-data)))
|
|
|
|
|
|
|
|
|
|
(is (= short-name-with-get (:metric-name short-data-get)))
|
|
|
|
|
(is (= 1 (:count short-data-get)))
|
|
|
|
|
(is (<= 5 (:mean short-data-get)))
|
|
|
|
|
(is (<= 5 (:aggregate short-data-get)))
|
|
|
|
|
|
|
|
|
|
(is (= short-name-with-post (:metric-name short-data-post)))
|
|
|
|
|
(is (= 1 (:count short-data-post)))
|
|
|
|
|
(is (<= 5 (:mean short-data-post)))
|
|
|
|
|
(is (<= 5 (:aggregate short-data-post)))
|
|
|
|
|
|
|
|
|
|
(is (>= 1 (Math/abs (- (:aggregate short-data)
|
|
|
|
|
(+ (:aggregate short-data-get)
|
|
|
|
|
(:aggregate short-data-post))))))
|
|
|
|
|
|
|
|
|
|
(is (= long-name (:metric-name long-data)))
|
|
|
|
|
(is (= 1 (:count long-data)))
|
|
|
|
|
(is (<= 100 (:mean long-data)))
|
|
|
|
|
(is (<= 100 (:aggregate long-data)))
|
|
|
|
|
|
|
|
|
|
(is (> (:mean long-data) (:mean short-data))))))))
|
|
|
|
|
(with-open [client (sync/create-client {})]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(testing "get-client-metric-registry returns nil if no metric registry passed in"
|
|
|
|
|
(is (= nil (common/get-client-metric-registry client))))))))))
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
|
|
|
|
|
(deftest java-metrics-for-unbuffered-streaming-test
|
|
|
|
|
(testlogging/with-test-logging
|
|
|
|
|
(let [data (unbuffered-test/generate-data (* 1024 1024))]
|
|
|
|
|
(testing "metrics work for a successful request"
|
|
|
|
|
(let [metric-registry (MetricRegistry.)]
|
|
|
|
|
(testwebserver/with-test-webserver-and-config
|
|
|
|
|
(unbuffered-test/successful-handler data nil) port {:shutdown-timeout-seconds 1}
|
|
|
|
|
(with-open [client (-> (ClientOptions.)
|
|
|
|
|
(.setSocketTimeoutMilliseconds 20000)
|
|
|
|
|
(.setConnectTimeoutMilliseconds 100)
|
|
|
|
|
(.setMetricRegistry metric-registry)
|
|
|
|
|
(Async/createClient))]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(let [url (str "http://localhost:" port "/hello")
|
|
|
|
|
request-options (doto (RequestOptions. url)
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
(.setAs ResponseBodyType/UNBUFFERED_STREAM))
|
|
|
|
|
response (-> client (.get request-options) .deref)
|
|
|
|
|
status (.getStatus response)
|
|
|
|
|
body (.getBody response)]
|
|
|
|
|
(is (= 200 status))
|
|
|
|
|
(let [instream body
|
|
|
|
|
buf (make-array Byte/TYPE 4)]
|
|
|
|
|
(.read instream buf)
|
|
|
|
|
(is (= "xxxx" (String. buf "UTF-8"))) ;; Make sure we can read a few chars off of the stream
|
2016-04-22 21:26:51 +00:00
|
|
|
(Thread/sleep 1000) ;; check that the full-response metric takes this into account
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
(is (= (str data "yyyy") (str "xxxx" (slurp instream))))) ;; Read the rest and validate
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(let [client-metric-registry (.getMetricRegistry client)
|
|
|
|
|
client-metrics (Metrics/getClientMetrics client-metric-registry)
|
|
|
|
|
client-metrics-data (Metrics/getClientMetricsData client-metric-registry)
|
2016-04-22 21:26:51 +00:00
|
|
|
full-response-name (format "%s.with-url.%s.full-response" metric-namespace url)
|
|
|
|
|
full-response-name-with-method (format "%s.with-url-and-method.%s.GET.full-response"
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
metric-namespace url)]
|
2016-04-22 21:26:51 +00:00
|
|
|
(is (= [full-response-name]
|
2016-05-05 11:48:07 +00:00
|
|
|
(map #(.getMetricName %) (.getUrlTimers client-metrics))
|
|
|
|
|
(map #(.getMetricName %) (.getUrlData client-metrics-data))))
|
2016-04-22 21:26:51 +00:00
|
|
|
(is (= [full-response-name-with-method]
|
2016-05-05 11:48:07 +00:00
|
|
|
(map #(.getMetricName %) (.getUrlAndMethodTimers client-metrics))
|
|
|
|
|
(map #(.getMetricName %) (.getUrlAndMethodData client-metrics-data))))
|
|
|
|
|
(is (= [] (.getMetricIdTimers client-metrics) (.getMetricIdData client-metrics-data)))
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(is (every? #(instance? ClientTimer %)
|
2016-05-05 11:48:07 +00:00
|
|
|
(concat (.getUrlTimers client-metrics)
|
|
|
|
|
(.getUrlAndMethodTimers client-metrics))))
|
|
|
|
|
(let [full-response-data (first (.getUrlData client-metrics-data))]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(is (every? #(instance? ClientMetricData %)
|
2016-05-05 11:48:07 +00:00
|
|
|
(concat (.getUrlData client-metrics-data)
|
|
|
|
|
(.getUrlAndMethodData client-metrics-data))))
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
|
2016-04-22 21:26:51 +00:00
|
|
|
(is (= 1 (.getCount full-response-data)))
|
|
|
|
|
(is (= full-response-name (.getMetricName full-response-data)))
|
|
|
|
|
(is (<= 1000 (.getMean full-response-data)))
|
|
|
|
|
(is (<= 1000 (.getAggregate full-response-data))))))))))
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
(testing "metrics work for failed request"
|
|
|
|
|
(try
|
|
|
|
|
(testwebserver/with-test-webserver-and-config
|
|
|
|
|
(unbuffered-test/blocking-handler data) port {:shutdown-timeout-seconds 1}
|
|
|
|
|
(let [metric-registry (MetricRegistry.)]
|
|
|
|
|
(with-open [client (-> (ClientOptions.)
|
|
|
|
|
(.setSocketTimeoutMilliseconds 200)
|
|
|
|
|
(.setConnectTimeoutMilliseconds 100)
|
|
|
|
|
(.setMetricRegistry metric-registry)
|
|
|
|
|
(Async/createClient))]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(let [url (str "http://localhost:" port "/hello")
|
|
|
|
|
request-options (doto (RequestOptions. url)
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
(.setAs ResponseBodyType/UNBUFFERED_STREAM))
|
|
|
|
|
response (-> client (.get request-options) .deref)
|
|
|
|
|
error (.getError response)
|
|
|
|
|
body (.getBody response)]
|
|
|
|
|
(is (nil? error))
|
|
|
|
|
(is (thrown? SocketTimeoutException (slurp body)))
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(let [client-metric-registry (.getMetricRegistry client)
|
|
|
|
|
client-metrics (Metrics/getClientMetrics client-metric-registry)
|
|
|
|
|
client-metrics-data (Metrics/getClientMetricsData client-metric-registry)
|
2016-04-22 21:26:51 +00:00
|
|
|
full-response-name (format "%s.with-url.%s.full-response" metric-namespace url)
|
|
|
|
|
full-response-name-with-method (format "%s.with-url-and-method.%s.GET.full-response"
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
metric-namespace url)]
|
2016-04-22 21:26:51 +00:00
|
|
|
(is (= [full-response-name]
|
2016-05-05 11:48:07 +00:00
|
|
|
(map #(.getMetricName %) (.getUrlTimers client-metrics))
|
|
|
|
|
(map #(.getMetricName %) (.getUrlData client-metrics-data))))
|
2016-04-22 21:26:51 +00:00
|
|
|
(is (= [full-response-name-with-method]
|
2016-05-05 11:48:07 +00:00
|
|
|
(map #(.getMetricName %) (.getUrlAndMethodTimers client-metrics))
|
|
|
|
|
(map #(.getMetricName %) (.getUrlAndMethodData client-metrics-data))))
|
|
|
|
|
(is (= [] (.getMetricIdTimers client-metrics)
|
|
|
|
|
(.getMetricIdData client-metrics-data)))
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(is (every? #(instance? ClientTimer %)
|
2016-05-05 11:48:07 +00:00
|
|
|
(concat (.getUrlTimers client-metrics)
|
|
|
|
|
(.getUrlAndMethodTimers client-metrics))))
|
|
|
|
|
(let [full-response-data (first (.getUrlData client-metrics-data))]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(is (every? #(instance? ClientMetricData %)
|
2016-05-05 11:48:07 +00:00
|
|
|
(concat (.getUrlData client-metrics-data)
|
|
|
|
|
(.getUrlAndMethodData client-metrics-data))))
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
|
2016-04-22 21:26:51 +00:00
|
|
|
(is (= 1 (.getCount full-response-data)))
|
|
|
|
|
(is (= full-response-name (.getMetricName full-response-data)))
|
|
|
|
|
(is (<= 200 (.getMean full-response-data)))
|
|
|
|
|
(is (<= 200 (.getAggregate full-response-data)))))))))
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
(catch TimeoutException e
|
|
|
|
|
;; Expected whenever a server-side failure is generated
|
|
|
|
|
))))))
|
|
|
|
|
|
|
|
|
|
(deftest clojure-metrics-for-unbuffered-streaming-test
|
|
|
|
|
(testlogging/with-test-logging
|
|
|
|
|
(let [data (unbuffered-test/generate-data (* 1024 1024))
|
|
|
|
|
opts {:as :unbuffered-stream}]
|
|
|
|
|
(testing "metrics work for a successful request"
|
|
|
|
|
(let [metric-registry (MetricRegistry.)]
|
|
|
|
|
(testwebserver/with-test-webserver-and-config
|
|
|
|
|
(unbuffered-test/successful-handler data nil) port {:shutdown-timeout-seconds 1}
|
|
|
|
|
(with-open [client (async/create-client {:connect-timeout-milliseconds 100
|
|
|
|
|
:socket-timeout-milliseconds 20000
|
|
|
|
|
:metric-registry metric-registry})]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(let [url (str "http://localhost:" port "/hello")
|
|
|
|
|
response @(common/get client url opts)
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
{:keys [status body]} response]
|
|
|
|
|
(is (= 200 status))
|
|
|
|
|
(let [instream body
|
|
|
|
|
buf (make-array Byte/TYPE 4)]
|
|
|
|
|
(.read instream buf)
|
|
|
|
|
(is (= "xxxx" (String. buf "UTF-8"))) ;; Make sure we can read a few chars off of the stream
|
2016-04-22 21:26:51 +00:00
|
|
|
(Thread/sleep 1000) ;; check that the full-response metric takes this into account
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
(is (= (str data "yyyy") (str "xxxx" (slurp instream))))) ;; Read the rest and validate
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(let [client-metric-registry (common/get-client-metric-registry client)
|
|
|
|
|
client-metrics (metrics/get-client-metrics client-metric-registry)
|
|
|
|
|
client-metrics-data (metrics/get-client-metrics-data client-metric-registry)
|
2016-04-22 21:26:51 +00:00
|
|
|
full-response-name (format "%s.with-url.%s.full-response" metric-namespace url)
|
|
|
|
|
full-response-name-with-method (format "%s.with-url-and-method.%s.GET.full-response"
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
metric-namespace url)]
|
2016-04-22 21:26:51 +00:00
|
|
|
(is (= [full-response-name]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(map #(.getMetricName %) (:url client-metrics))
|
|
|
|
|
(map :metric-name (:url client-metrics-data))))
|
2016-04-22 21:26:51 +00:00
|
|
|
(is (= [full-response-name-with-method]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(map #(.getMetricName %) (:url-and-method client-metrics))
|
|
|
|
|
(map :metric-name (:url-and-method client-metrics-data))))
|
|
|
|
|
(is (= [] (:metric-id client-metrics) (:metric-id client-metrics-data)))
|
|
|
|
|
(is (every? #(instance? ClientTimer %)
|
|
|
|
|
(concat (:url client-metrics)
|
|
|
|
|
(:url-and-method client-metrics))))
|
2016-04-22 21:26:51 +00:00
|
|
|
(let [full-response-data (first (:url client-metrics-data))]
|
|
|
|
|
(is (= {:count 1 :metric-name full-response-name}
|
|
|
|
|
(select-keys full-response-data [:metric-name :count])))
|
|
|
|
|
(is (<= 1000 (:mean full-response-data)))
|
|
|
|
|
(is (<= 1000 (:aggregate full-response-data))))))))))
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
(testing "metrics work for a failed request"
|
|
|
|
|
(try
|
|
|
|
|
(testwebserver/with-test-webserver-and-config
|
|
|
|
|
(unbuffered-test/blocking-handler data) port {:shutdown-timeout-seconds 1}
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(let [metric-registry (MetricRegistry.)
|
|
|
|
|
url (str "http://localhost:" port "/hello")]
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
(with-open [client (async/create-client {:connect-timeout-milliseconds 100
|
|
|
|
|
:socket-timeout-milliseconds 200
|
|
|
|
|
:metric-registry metric-registry})]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(let [response @(common/get client url opts)
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
{:keys [body error]} response]
|
|
|
|
|
(is (nil? error))
|
|
|
|
|
;; Consume the body to get the exception
|
|
|
|
|
(is (thrown? SocketTimeoutException (slurp body))))
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(let [client-metric-registry (common/get-client-metric-registry client)
|
|
|
|
|
client-metrics (metrics/get-client-metrics client-metric-registry)
|
|
|
|
|
client-metrics-data (metrics/get-client-metrics-data client-metric-registry)
|
2016-04-22 21:26:51 +00:00
|
|
|
full-response-name (format "%s.with-url.%s.full-response" metric-namespace url)
|
|
|
|
|
full-response-name-with-method (format "%s.with-url-and-method.%s.GET.full-response"
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
metric-namespace url)]
|
2016-04-22 21:26:51 +00:00
|
|
|
(is (= [full-response-name]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(map #(.getMetricName %) (:url client-metrics))
|
|
|
|
|
(map :metric-name (:url client-metrics-data))))
|
2016-04-22 21:26:51 +00:00
|
|
|
(is (= [full-response-name-with-method]
|
(TK-316) Improve metrics API
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
2016-04-07 00:17:53 +00:00
|
|
|
(map #(.getMetricName %) (:url-and-method client-metrics))
|
|
|
|
|
(map :metric-name (:url-and-method client-metrics-data))))
|
|
|
|
|
(is (= [] (:metric-id client-metrics) (:metric-id client-metrics-data)))
|
|
|
|
|
(is (every? #(instance? ClientTimer %)
|
|
|
|
|
(concat (:url client-metrics)
|
|
|
|
|
(:url-and-method client-metrics))))
|
2016-04-22 21:26:51 +00:00
|
|
|
(let [full-response-data (first (:url client-metrics-data))]
|
|
|
|
|
(is (= {:count 1 :metric-name full-response-name}
|
|
|
|
|
(select-keys full-response-data [:metric-name :count])))
|
|
|
|
|
(is (<= 200 (:mean full-response-data)))
|
|
|
|
|
(is (<= 200 (:aggregate full-response-data))))))))
|
(TK-316) Add metrics support
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
2016-02-26 05:39:21 +00:00
|
|
|
(catch TimeoutException e
|
|
|
|
|
;; Expected whenever a server-side failure is generated
|
|
|
|
|
))))))
|
2016-09-30 22:35:07 +00:00
|
|
|
|
|
|
|
|
(deftest metric-namespace-test
|
|
|
|
|
(let [metric-prefix "my-metric-prefix"
|
|
|
|
|
server-id "my-server"
|
|
|
|
|
metric-name-with-prefix
|
|
|
|
|
(format "%s.http-client.experimental.with-url.%s.full-response" metric-prefix hello-url)
|
|
|
|
|
metric-name-with-server-id
|
|
|
|
|
(format "puppetlabs.%s.http-client.experimental.with-url.%s.full-response"
|
|
|
|
|
server-id hello-url)
|
|
|
|
|
get-metric-name (fn [metric-registry]
|
|
|
|
|
(.getMetricName (first (Metrics/getClientMetricsDataByUrl
|
|
|
|
|
metric-registry hello-url))))]
|
|
|
|
|
(testlogging/with-test-logging
|
|
|
|
|
(testutils/with-app-with-config
|
|
|
|
|
app
|
|
|
|
|
[jetty9/jetty9-service test-metric-web-service]
|
|
|
|
|
{:webserver {:port 10000}}
|
|
|
|
|
(testing "custom metric namespace works for java async client"
|
|
|
|
|
(testing "metric prefix works"
|
|
|
|
|
(let [metric-registry (MetricRegistry.)]
|
|
|
|
|
(with-open [client-with-metric-prefix (Async/createClient
|
|
|
|
|
(doto (ClientOptions.)
|
|
|
|
|
(.setMetricRegistry metric-registry)
|
|
|
|
|
(.setMetricPrefix metric-prefix)))]
|
|
|
|
|
(is (= (format "%s.http-client.experimental" metric-prefix)
|
|
|
|
|
(.getMetricNamespace client-with-metric-prefix)))
|
|
|
|
|
(-> client-with-metric-prefix (.get (RequestOptions. hello-url)))
|
|
|
|
|
(is (= metric-name-with-prefix (get-metric-name metric-registry))))))
|
|
|
|
|
(testing "server id works"
|
|
|
|
|
(let [metric-registry (MetricRegistry.)]
|
|
|
|
|
(with-open [client-with-server-id (Async/createClient
|
|
|
|
|
(doto (ClientOptions.)
|
|
|
|
|
(.setMetricRegistry metric-registry)
|
|
|
|
|
(.setServerId server-id)))]
|
|
|
|
|
(-> client-with-server-id (.get (RequestOptions. hello-url)))
|
|
|
|
|
(is (= metric-name-with-server-id (get-metric-name metric-registry))))))
|
|
|
|
|
(testing "metric prefix overrides server id if both are set"
|
|
|
|
|
(let [metric-registry (MetricRegistry.)]
|
|
|
|
|
(with-open [client-with-server-id (Async/createClient
|
|
|
|
|
(doto (ClientOptions.)
|
|
|
|
|
(.setMetricRegistry metric-registry)
|
|
|
|
|
(.setMetricPrefix metric-prefix)
|
|
|
|
|
(.setServerId server-id)))]
|
|
|
|
|
(-> client-with-server-id (.get (RequestOptions. hello-url)))
|
|
|
|
|
(is (= metric-name-with-prefix (get-metric-name metric-registry)))
|
|
|
|
|
(is (logged? #"Metric prefix and server id both set.*" :warn))))))
|
|
|
|
|
(testing "custom metric namespace works for clojure async client"
|
|
|
|
|
(testing "metric prefix works"
|
|
|
|
|
(let [metric-registry (MetricRegistry.)]
|
|
|
|
|
(with-open [client-with-metric-prefix (async/create-client
|
|
|
|
|
{:metric-registry metric-registry
|
|
|
|
|
:metric-prefix metric-prefix})]
|
|
|
|
|
(is (= (format "%s.http-client.experimental" metric-prefix)
|
|
|
|
|
(common/get-client-metric-namespace client-with-metric-prefix)))
|
|
|
|
|
(-> client-with-metric-prefix (common/get hello-url))
|
|
|
|
|
(is (= metric-name-with-prefix (get-metric-name metric-registry))))))
|
|
|
|
|
(testing "server id works"
|
|
|
|
|
(let [metric-registry (MetricRegistry.)]
|
|
|
|
|
(with-open [client-with-server-id (async/create-client
|
|
|
|
|
{:metric-registry metric-registry
|
|
|
|
|
:server-id server-id})]
|
|
|
|
|
(-> client-with-server-id (common/get hello-url))
|
|
|
|
|
(is (= metric-name-with-server-id (get-metric-name metric-registry))))))
|
|
|
|
|
(testing "metric prefix overrides server id if both are set"
|
|
|
|
|
(let [metric-registry (MetricRegistry.)]
|
|
|
|
|
(with-open [client-with-server-id (async/create-client
|
|
|
|
|
{:metric-registry metric-registry
|
|
|
|
|
:metric-prefix metric-prefix
|
|
|
|
|
:server-id server-id})]
|
|
|
|
|
(-> client-with-server-id (common/get hello-url))
|
|
|
|
|
(is (= metric-name-with-prefix (get-metric-name metric-registry)))))))
|
|
|
|
|
(testing "custom metric namespace works for Java sync client"
|
|
|
|
|
(testing "metric prefix works"
|
|
|
|
|
(let [metric-registry (MetricRegistry.)]
|
|
|
|
|
(with-open [client-with-metric-prefix (Sync/createClient
|
|
|
|
|
(doto (ClientOptions.)
|
|
|
|
|
(.setMetricRegistry metric-registry)
|
|
|
|
|
(.setMetricPrefix metric-prefix)))]
|
|
|
|
|
(is (= (format "%s.http-client.experimental" metric-prefix)
|
|
|
|
|
(.getMetricNamespace client-with-metric-prefix)))
|
|
|
|
|
(-> client-with-metric-prefix (.get (RequestOptions. hello-url)))
|
|
|
|
|
(is (= metric-name-with-prefix (get-metric-name metric-registry))))))
|
|
|
|
|
(testing "server id works"
|
|
|
|
|
(let [metric-registry (MetricRegistry.)]
|
|
|
|
|
(with-open [client-with-server-id (Sync/createClient
|
|
|
|
|
(doto (ClientOptions.)
|
|
|
|
|
(.setMetricRegistry metric-registry)
|
|
|
|
|
(.setServerId server-id)))]
|
|
|
|
|
(-> client-with-server-id (.get (RequestOptions. hello-url)))
|
|
|
|
|
(is (= metric-name-with-server-id (get-metric-name metric-registry))))))
|
|
|
|
|
(testing "metric prefix overrides server id if both are set"
|
|
|
|
|
(let [metric-registry (MetricRegistry.)]
|
|
|
|
|
(with-open [client-with-server-id (Sync/createClient
|
|
|
|
|
(doto (ClientOptions.)
|
|
|
|
|
(.setMetricRegistry metric-registry)
|
|
|
|
|
(.setMetricPrefix metric-prefix)
|
|
|
|
|
(.setServerId server-id)))]
|
|
|
|
|
(-> client-with-server-id (.get (RequestOptions. hello-url)))
|
|
|
|
|
(is (= metric-name-with-prefix (get-metric-name metric-registry)))))))
|
|
|
|
|
(testing "custom metric namespace works for clojure sync client"
|
|
|
|
|
(testing "metric prefix works"
|
|
|
|
|
(let [metric-registry (MetricRegistry.)]
|
|
|
|
|
(with-open [client-with-metric-prefix (sync/create-client
|
|
|
|
|
{:metric-registry metric-registry
|
|
|
|
|
:metric-prefix metric-prefix})]
|
|
|
|
|
(is (= (format "%s.http-client.experimental" metric-prefix)
|
|
|
|
|
(common/get-client-metric-namespace client-with-metric-prefix)))
|
|
|
|
|
(-> client-with-metric-prefix (common/get hello-url))
|
|
|
|
|
(is (= metric-name-with-prefix (get-metric-name metric-registry))))))
|
|
|
|
|
(testing "server id works"
|
|
|
|
|
(let [metric-registry (MetricRegistry.)]
|
|
|
|
|
(with-open [client-with-server-id (sync/create-client
|
|
|
|
|
{:metric-registry metric-registry
|
|
|
|
|
:server-id server-id})]
|
|
|
|
|
(-> client-with-server-id (common/get hello-url))
|
|
|
|
|
(is (= metric-name-with-server-id (get-metric-name metric-registry))))))
|
|
|
|
|
(testing "metric prefix overrides server id if both are set"
|
|
|
|
|
(let [metric-registry (MetricRegistry.)]
|
|
|
|
|
(with-open [client-with-server-id (sync/create-client
|
|
|
|
|
{:metric-registry metric-registry
|
|
|
|
|
:metric-prefix metric-prefix
|
|
|
|
|
:server-id server-id})]
|
|
|
|
|
(-> client-with-server-id (common/get hello-url))
|
|
|
|
|
(is (= metric-name-with-prefix (get-metric-name metric-registry)))))))))))
|