This commit passes the negotiated locale via the i18n/*locale* variable
through the Accept-Language header by default in a clj-http-client
request (only in the Clojure client, not the Java client).
Add two new client options - `server-id` and `metric-prefix` that allow the
metric namespace to be configured rather than always
`puppetlabs.http-client.experimental`.
If `server-id` is set, the metric namespace becomes
`puppetlabs.<server-id>.http-client.experimental`.
If `metric-prefix` is set, the metric namespace becomes
`<metric-prefix>.http-client.experimental`.
If both are set, `metric-prefix` wins out and a warning message is logged.
Also add a `get-client-metric-namespace`/`getMetricNamespace` method on the
client (clojure and java) to get back the metric namespace.
(MAINT) remove comments
(MAINT) change body message to reflect 400 status
(MAINT) change test to use Java and Clj API
(MAINT) refactor RequestOptions in let block of java api test
(MAINT) add tests for async cookie support
When constructing url timers for request metrics, we strip off any username,
password, query params, and path fragments on the request uri. This commit
provides a public method to do this conversion of a url, so that users can
take the url they used for their request and easily figure out the url we used
for the metric.
Previously, if any of the `get-client-metrics*` methods (in Clojure or Java)
had a nil metric registry passed into them, they would return nil. This commit
updates the behavior of all of these methods to instead return an error - a
schema error (clojure.lang.ExceptionInfo) for the clojure functions, and an
IllegalArgumentException for the java methods.
Previously, all the get-client-metrics* functions had two arities - one with a
metric type and one without. This commit removes the arity with a metric type
from all these functions since currently we only have one metric type and
since we aren't entirely certain of what we want the API to be when we add
additional metric types. When we add more metric types, we will change the API
in a backward compatible way.
Rather than having e.g a nil url and method in the metric data when metric-id
is filtered on, or a nil metric id if url and method are filtered on, use
specific schemas for metrics data returned for each metric category - metric
id, url, and url and method.
Rename the Metric Types from `bytes-read`/`init-repsonse` to
`full-response`/`initial-response` (note that the `initial-response` timer has
not yet been implemented, but when it is this will be its name).
This commit does several things to improve the metrics API:
* Move get-client-metrics(-data) functions off of client:
Previously, the `get-client-metrics(-data)`/`.getClientMetrics(Data)`
functions were on the client. This didn't entirely make sense because if two
clients were using the same metric registry, these functions would actually
return metrics data for *all* clients, rather than just for the client the
function was called on. This commit removes these functions and changes the
tests to use the metrics namepsace/Metrics class versions instead, which take
a metric registry return all http-client related metrics on that metric
registry.
* Add a `with-url-and-method` namespace:
Move the timers that have both the url and the method from the `with-url`
namespace to a new `with-url-and-method` namepsace. This better matches the
data structure are returned in and the filtering mechanisms for getting them
back out (discussed below), and also removes a slight chance of having a
conflict with a url timer.
* Add a ClientTimer class:
Add a ClientTimer class that wraps the Timer class and also holds onto the
url, method, or metric id used to create the timer. Use this to add url,
method, metricId to ClientMetricData.
* Change the `getClientMetrics(Data)` methods to return a map of metric
category to an array of timers or timer data:
Previously, the methods for getting http client metrics/metric data out of a
metric registry returned a map of metric name to timer instance or metric
data. However, the metric name was most likely not useful to users, who
probably just want to iterate through the timers/data. This commit makes the
output of these functions more useful by returning arrays of timers/data
sorted into a map indexed by metric category (url, url and method, or metric
id).
* Add `getClientMetricsBy*` methods:
Add `getClientMetrics(Data)ByUrl`, `getClientMetrics(Data)ByUrlAndMethod`, and
`getClientMetrics(Data)ByMetricId` methods (and clojure versions) that allow
for filtering by a specific metric category and return an array of timers or
timer data that match the url, url and method, or metric id specified.
* Remove the filter-builder functions:
Previously, the `get-client-metrics(-data)` functions did filtering by taking
a filter map, and there were filter-builder functions to build these filter
maps. Now that there are separate filtering methods, these filter maps are no
longer used and the filter-builder functions are removed.
This commit adds metrics support to the http client (clojure and java, sync
and async). A metric registry can optionally be passed into the client as a
client option on creation. If a metric registry is present, timers will be
added to time each request.
By default, a timer is added for the URL (stripped of username, password,
query string, and path fragments) and the URL plus the method used for the
request. In addition, a request can include a `metric-id` option, which takes
a tuple of metric ids. If this request option is specified, a timer will be
created for each element of the metric id tuple - thus if the tuple is [:foo
:bar :baz] there will be a foo timer, a foo.bar timer, and a foo.bar.baz
timer.
In addition, each timer has a "MetricType" - currently there is only one
metric type, bytes-read, which is stopped when the full response has been
read. In the future, we may add "response-init" timers that get stopped when
the first byte of the response has been read.
This commit also adds a `get-client-metrics`/`.getClientMetrics` function that
takes a client instance and returns the http client-specific metrics from the
metric registry and a `get-client-metrics-data`/`.getClientMetricsData`
function for clojure and java sync and async clients to get out metrics data
from the client. This function takes a client instance and returns a map of
metric name to a map of metric data (for clojure) or a ClientMetricData object
(for java), both of which include the mean, count, and aggregate for the timer
These `get-client-metrics*`/`.getClientMetrics*` functions also have versions
that take a url, url and method, or metric id to allow for filtering of the
timers/metrics data returned by these functions.
The clojure versions of these functions take a metric filter map. There are
also metric filter builder functions to build up the type of metric filter
desired from a url, a url and method, or a metric id. These will prevent users
from having to know the specifics of how to build a metric themselves; instead
they can use a convenience function.
An empty metric id can be passed in to the filter to return all metric-id
timers.
These functions were previously used for initializing an ssl context for the
clojure http client. However, with the clojure client initialization now just
wrapping the java code, these are no longer needed.
This commit removes duplicate functionality from the Clojure implementation and reuses the Java request/response handling impl. It doesn't address client creation so there are still two distinct paths for that.
The main changes are Ring <-> Java request/response conversion and the addition of interface ResponseDeliveryDelegate. The delegate implementations take care of the language-specific aspects of response building and promise delivery. One of the Clojure test namespaces is removed too.
There are a few places where we map request method keywords to
http-client function. The http client then maps them back to keywords
which is redundant. This adds a general request function to the protocol
that accepts a keyword method.
This commit updates us to the latest version of the Apache
HTTPAsyncClient. This is necessary in order to make it
possible to use this library in the same VM as the latest
version of clj-http, because they have some common dependencies.
The commit also cleans up some other dependencies while I was
in there.
- Fixes to ExceptionInsertingPipedInputStream
- Fix to async consumer and removal of decompression code
- Use ResponseContentEncoding to do all decompression including removing previous approach
- Clarify tests and add ConnectionException
Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
This commit updates the short connect timeout tests to tolerate either a
ConnectException or SocketTimeoutException as a success condition.
Connections could fail with either exception depending upon timing or
network setup.
This commit enables http clients to optionally configure a connect
and/or socket timeout for requests. For persistent clients, the
timeout values can only be configured at the client and not a
per-request level. Non-persistent client requests support the
configuration of these timeouts.
This commit also bumps a few dependencies - tk-jetty9 to 1.2.0,
trapperkeeper to 1.10, ssl-utils to 0.8.0, and clojure to 1.6.0 -
and adds the 'test' tk-jetty9 library as a dev dependency for
testing.
Fix a bug in which a default charset of UTF-8 would be added to the
Content-Type header for certain requests. This change restricts that
behavior to cases in which the body is a string and there is no charset
provided by the caller.
Fix a bug in which a default charset of UTF-8 would be added to the
Content-Type header for certain requests. This change restricts that
behavior to cases in which the body is a string and there is no charset
provided by the caller.
This commit restores several of the `rediret-test-async` tests, using a
client to make the requests since the client-less single-request async
APIs are no longer available.
This commit restores the `query-params-test-async` tests, using a client
to make the requests since the client-less single-request async APIs are
no longer available.
This commit removes the `AsyncClose` class and references to it. These
changes are in preparation for making the close call after the response
has been dereferenced - to be done in a later commit. The `persistent`
flag/option was also removed from request handling as it will no longer
be used for determining whether or not the client should be closed
following a request.
Add a PersistentSyncHttpClient class, which is a wrapper around
CloseableHttpAsyncClient and allows multiple requests to be made
with a persistent synchronous client.
Separate the Java RequestOptions class into two new classes:
RequestOptions, which contains options related to the request,
and ClientOptions, which contains options related to the client.
Both are now required when calling the request methods in
AsyncHttpClient and SyncHttpClient.
The test code related to the configurable SSL protocols was
written using some ciphers that are available in OpenJDK, but
are not available by default in Oracle JDK. This caused the
tests to fail on Oracle JDKs.
This commit changes the test to use some older ciphers that
are available in the Oracle JDK out of the box.
This commit moves each close call made on a `CloseableHttpAsyncClient`
from the thread on which the request callback is made over to a
different thread in order to avoid a deadlock.
Previously, the client close call was made in the context of thread from
which the callback was invoked. The Apache HTTP async client library
makes these calls from one of its i/o reactor threads. In the case of a
"failed" request, e.g., failure to make the client connection, this call
occurs in the context of the primary i/o reactor thread. The client
close call does an indefinite join on the primary i/o reactor thread
and, therefore, this call causes a deadlock. The client never closes
and all resources associated with it - threads, file descriptors, etc. -
remain open until the process is shutdown.
This commit introduces a new class, `AsyncClose`, which manages a small,
fixed size thread pool from which client close calls can be executed.
The `async` Clojure and `JavaClient` APIs were updated to use `AsyncClose`
whenever a client needs to be closed from a callback.
This commit adds configuration settings for the SSL protocols
and cipher suites, in both the java and clojure clients. It
also adds a list of default protocols which will be used if
the protocols setting is not explicitly set.
