yogsototh/clj-http-client
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
clj-http-client/test/puppetlabs/http/client/async_plaintext_test.clj

521 lines
27 KiB
Clojure
Raw Normal View History

(TK-23) Port to apache HttpAsyncClient * Ports the code over to use the Apache HttpAsyncClient library instead of http-kit, as we were getting some weird and inconsistent SSL errors from http-kit, and * Simplify the API (by eliminating a lot of the supported configuration options). Some of these options we'll eventually want to add back in, but for now, getting rid of them makes the API less cluttered and also gives us the opportunity to do a better job writing tests for new options when we do add support for them. For now it only supports constructing a new client on every request. I intend to add API for creating a client explicitly, and a protocol for how to interact with the client. This will allow users to re-use an existing client as they see fit... but this hasn't been implemented yet.
2014-07-03 15:58:46 +00:00
(ns puppetlabs.http.client.async-plaintext-test
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
(:import (com.puppetlabs.http.client Async RequestOptions ClientOptions ResponseBodyType)
(TK-29) Move to URI instead of String Have the RequestOptions constructor take a URI instead of a string. Add signature to the HTTP request methods that take a URI.
2014-08-08 23:32:48 +00:00
(org.apache.http.impl.nio.client HttpAsyncClients)
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
(java.net URI SocketTimeoutException ServerSocket ConnectException)
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
(java.io PipedInputStream PipedOutputStream)
Rework tests for improved clarity and AsyncConsumer releaseResources() fix
2015-10-13 10:40:32 +00:00
(java.util.concurrent TimeoutException)
(java.util UUID))
(TK-23) Port to apache HttpAsyncClient * Ports the code over to use the Apache HttpAsyncClient library instead of http-kit, as we were getting some weird and inconsistent SSL errors from http-kit, and * Simplify the API (by eliminating a lot of the supported configuration options). Some of these options we'll eventually want to add back in, but for now, getting rid of them makes the API less cluttered and also gives us the opportunity to do a better job writing tests for new options when we do add support for them. For now it only supports constructing a new client on every request. I intend to add API for creating a client explicitly, and a protocol for how to interact with the client. This will allow users to re-use an existing client as they see fit... but this hasn't been implemented yet.
2014-07-03 15:58:46 +00:00
(:require [clojure.test :refer :all]
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
[clojure.java.io :as io]
(TK-29) Add query params to Java client Add query params back into the Java client, along with tests to validate this functionality is working.
2014-08-06 17:49:48 +00:00
[puppetlabs.http.client.test-common :refer :all]
(TK-23) Port to apache HttpAsyncClient * Ports the code over to use the Apache HttpAsyncClient library instead of http-kit, as we were getting some weird and inconsistent SSL errors from http-kit, and * Simplify the API (by eliminating a lot of the supported configuration options). Some of these options we'll eventually want to add back in, but for now, getting rid of them makes the API less cluttered and also gives us the opportunity to do a better job writing tests for new options when we do add support for them. For now it only supports constructing a new client on every request. I intend to add API for creating a client explicitly, and a protocol for how to interact with the client. This will allow users to re-use an existing client as they see fit... but this hasn't been implemented yet.
2014-07-03 15:58:46 +00:00
[puppetlabs.trapperkeeper.core :as tk]
[puppetlabs.trapperkeeper.testutils.bootstrap :as testutils]
[puppetlabs.trapperkeeper.testutils.logging :as testlogging]
(TK-134) Add connect and socket timeout client configuration This commit enables http clients to optionally configure a connect and/or socket timeout for requests. For persistent clients, the timeout values can only be configured at the client and not a per-request level. Non-persistent client requests support the configuration of these timeouts. This commit also bumps a few dependencies - tk-jetty9 to 1.2.0, trapperkeeper to 1.10, ssl-utils to 0.8.0, and clojure to 1.6.0 - and adds the 'test' tk-jetty9 library as a dev dependency for testing.
2015-03-16 17:50:09 +00:00
[puppetlabs.trapperkeeper.testutils.webserver :as testwebserver]
(TK-23) Port to apache HttpAsyncClient * Ports the code over to use the Apache HttpAsyncClient library instead of http-kit, as we were getting some weird and inconsistent SSL errors from http-kit, and * Simplify the API (by eliminating a lot of the supported configuration options). Some of these options we'll eventually want to add back in, but for now, getting rid of them makes the API less cluttered and also gives us the opportunity to do a better job writing tests for new options when we do add support for them. For now it only supports constructing a new client on every request. I intend to add API for creating a client explicitly, and a protocol for how to interact with the client. This will allow users to re-use an existing client as they see fit... but this hasn't been implemented yet.
2014-07-03 15:58:46 +00:00
[puppetlabs.trapperkeeper.services.webserver.jetty9-service :as jetty9]
(TK-27) Refactor client protocols Refactor the async and sync client protocols into a single protocol which resides in a new file, common.clj.
2014-07-15 23:46:20 +00:00
[puppetlabs.http.client.common :as common]
(TK-23) Port to apache HttpAsyncClient * Ports the code over to use the Apache HttpAsyncClient library instead of http-kit, as we were getting some weird and inconsistent SSL errors from http-kit, and * Simplify the API (by eliminating a lot of the supported configuration options). Some of these options we'll eventually want to add back in, but for now, getting rid of them makes the API less cluttered and also gives us the opportunity to do a better job writing tests for new options when we do add support for them. For now it only supports constructing a new client on every request. I intend to add API for creating a client explicitly, and a protocol for how to interact with the client. This will allow users to re-use an existing client as they see fit... but this hasn't been implemented yet.
2014-07-03 15:58:46 +00:00
[puppetlabs.http.client.async :as async]
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
[schema.test :as schema-test]
[clojure.tools.logging :as log]))
(TK-23) Port to apache HttpAsyncClient * Ports the code over to use the Apache HttpAsyncClient library instead of http-kit, as we were getting some weird and inconsistent SSL errors from http-kit, and * Simplify the API (by eliminating a lot of the supported configuration options). Some of these options we'll eventually want to add back in, but for now, getting rid of them makes the API less cluttered and also gives us the opportunity to do a better job writing tests for new options when we do add support for them. For now it only supports constructing a new client on every request. I intend to add API for creating a client explicitly, and a protocol for how to interact with the client. This will allow users to re-use an existing client as they see fit... but this hasn't been implemented yet.
2014-07-03 15:58:46 +00:00
(use-fixtures :once schema-test/validate-schemas)
(defn app
(TK-134) Add connect and socket timeout client configuration This commit enables http clients to optionally configure a connect and/or socket timeout for requests. For persistent clients, the timeout values can only be configured at the client and not a per-request level. Non-persistent client requests support the configuration of these timeouts. This commit also bumps a few dependencies - tk-jetty9 to 1.2.0, trapperkeeper to 1.10, ssl-utils to 0.8.0, and clojure to 1.6.0 - and adds the 'test' tk-jetty9 library as a dev dependency for testing.
2015-03-16 17:50:09 +00:00
[_]
(TK-23) Port to apache HttpAsyncClient * Ports the code over to use the Apache HttpAsyncClient library instead of http-kit, as we were getting some weird and inconsistent SSL errors from http-kit, and * Simplify the API (by eliminating a lot of the supported configuration options). Some of these options we'll eventually want to add back in, but for now, getting rid of them makes the API less cluttered and also gives us the opportunity to do a better job writing tests for new options when we do add support for them. For now it only supports constructing a new client on every request. I intend to add API for creating a client explicitly, and a protocol for how to interact with the client. This will allow users to re-use an existing client as they see fit... but this hasn't been implemented yet.
2014-07-03 15:58:46 +00:00
{:status 200
:body "Hello, World!"})
(tk/defservice test-web-service
[[:WebserverService add-ring-handler]]
(init [this context]
(add-ring-handler app "/hello")
context))
(TK-27) Move test Move persistent async client test back into the async-plaintext-test file.
2014-07-15 22:07:43 +00:00
(deftest persistent-async-client-test
(testlogging/with-test-logging
(testutils/with-app-with-config app
[jetty9/jetty9-service test-web-service]
{:webserver {:port 10000}}
(TK-108) Add a persistent async Java client Add a new class, PersistentAsyncHttpClient, that is a wrapped CloseableHttpAsyncClient that is persistent.
2014-11-07 21:29:17 +00:00
(testing "java async client"
(let [request-options (RequestOptions. (URI. "http://localhost:10000/hello/"))
client-options (ClientOptions.)
(TK-108) Return interface in createClient methods Return interfaces in the createClient methods rather than the implementation classes themselves.
2014-11-10 19:24:36 +00:00
client (Async/createClient client-options)]
(TK-108) Add a persistent async Java client Add a new class, PersistentAsyncHttpClient, that is a wrapped CloseableHttpAsyncClient that is persistent.
2014-11-07 21:29:17 +00:00
(testing "HEAD request with persistent async client"
(let [response (.head client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= nil (.getBody (.deref response))))))
(testing "GET request with persistent async client"
(let [response (.get client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= "Hello, World!" (slurp (.getBody (.deref response)))))))
(testing "POST request with persistent async client"
(let [response (.post client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= "Hello, World!" (slurp (.getBody (.deref response)))))))
(testing "PUT request with persistent async client"
(let [response (.put client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= "Hello, World!" (slurp (.getBody (.deref response)))))))
(testing "DELETE request with persistent async client"
(let [response (.delete client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= "Hello, World!" (slurp (.getBody (.deref response)))))))
(testing "TRACE request with persistent async client"
(let [response (.trace client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= "Hello, World!" (slurp (.getBody (.deref response)))))))
(testing "OPTIONS request with persistent async client"
(let [response (.options client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= "Hello, World!" (slurp (.getBody (.deref response)))))))
(testing "PATCH request with persistent async client"
(let [response (.patch client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= "Hello, World!" (slurp (.getBody (.deref response)))))))
(TK-103) Restore client close tests This commit restores some of the client close tests which had been disabled temporarily per some of the refactoring for TK-108.
2014-11-12 22:20:50 +00:00
(testing "client closes properly"
(.close client)
(is (thrown? IllegalStateException
(.get client request-options))))))
(TK-108) Add a persistent async Java client Add a new class, PersistentAsyncHttpClient, that is a wrapped CloseableHttpAsyncClient that is persistent.
2014-11-07 21:29:17 +00:00
(testing "clojure async client"
(let [client (async/create-client {})]
(testing "HEAD request with persistent async client"
(let [response (common/head client "http://localhost:10000/hello/")]
(is (= 200 (:status @response)))
(is (= nil (:body @response)))))
(testing "GET request with persistent async client"
(let [response (common/get client "http://localhost:10000/hello/")]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(testing "POST request with persistent async client"
(let [response (common/post client "http://localhost:10000/hello/")]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(testing "PUT request with persistent async client"
(let [response (common/put client "http://localhost:10000/hello/")]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(testing "DELETE request with persistent async client"
(let [response (common/delete client "http://localhost:10000/hello/")]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(testing "TRACE request with persistent async client"
(let [response (common/trace client "http://localhost:10000/hello/")]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(testing "OPTIONS request with persistent async client"
(let [response (common/options client "http://localhost:10000/hello/")]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(testing "PATCH request with persistent async client"
(let [response (common/patch client "http://localhost:10000/hello/")]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(testing "client closes properly"
(common/close client)
(TK-103) Restore client close tests This commit restores some of the client close tests which had been disabled temporarily per some of the refactoring for TK-108.
2014-11-12 22:20:50 +00:00
(is (thrown? IllegalStateException
(common/get client
"http://localhost:10000/hello/")))))))))
(TK-27) Add tests for request-with-client Add asynchronous and synchronous tests for request-with-client.
2014-07-17 21:03:39 +00:00
(deftest request-with-client-test
(testlogging/with-test-logging
(testutils/with-app-with-config app
[jetty9/jetty9-service test-web-service]
{:webserver {:port 10000}}
(let [client (HttpAsyncClients/createDefault)
opts {:method :get :url "http://localhost:10000/hello/"}]
(.start client)
(testing "GET request works with request-with-client"
(let [response (async/request-with-client opts nil client)]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(testing "Client persists when passed to request-with-client"
(let [response (async/request-with-client opts nil client)]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(TK-29) Add query params to Java client Add query params back into the Java client, along with tests to validate this functionality is working.
2014-08-06 17:49:48 +00:00
(.close client)))))
(TK-103) Restored query-params-test-async tests This commit restores the `query-params-test-async` tests, using a client to make the requests since the client-less single-request async APIs are no longer available.
2014-11-14 23:36:40 +00:00
(deftest query-params-test-async
(testlogging/with-test-logging
(testutils/with-app-with-config app
[jetty9/jetty9-service test-params-web-service]
{:webserver {:port 8080}}
(testing "URL Query Parameters work with the Java client"
(let [client (Async/createClient (ClientOptions.))]
(try
(let [request-options (RequestOptions.
(URI. "http://localhost:8080/params?foo=bar&baz=lux"))
response (.get client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= queryparams (read-string (slurp (.getBody
(.deref response)))))))
(finally
(.close client)))))
(testing "URL Query Parameters work with the clojure client"
(with-open [client (async/create-client {})]
(let [opts {:method :get
:url "http://localhost:8080/params/"
:query-params queryparams
:as :text}
response (common/get client "http://localhost:8080/params" opts)]
(is (= 200 (:status @response)))
(is (= queryparams (read-string (:body @response)))))))
(testing "URL Query Parameters can be set directly in the URL"
(with-open [client (async/create-client {})]
(let [response (common/get client
"http://localhost:8080/params?paramone=one"
{:as :text})]
(is (= 200 (:status @response)))
(is (= (str {"paramone" "one"}) (:body @response))))))
(testing (str "URL Query Parameters set in URL are overwritten if params "
"are also specified in options map")
(with-open [client (async/create-client {})]
(let [response (common/get client
"http://localhost:8080/params?paramone=one&foo=lux"
query-options)]
(is (= 200 (:status @response)))
(is (= queryparams (read-string (:body @response))))))))))
(PE-5623) Add :force-redirects to clj client Add a :force-redirects option to the clojure client that will relax the redirect restrictions put on the client for a POST or PUT method.
2014-08-25 18:15:22 +00:00
(deftest redirect-test-async
(testlogging/with-test-logging
(testutils/with-app-with-config app
[jetty9/jetty9-service redirect-web-service]
{:webserver {:port 8080}}
(TK-103) Restored redirect-test-async tests This commit restores several of the `rediret-test-async` tests, using a client to make the requests since the client-less single-request async APIs are no longer available.
2014-11-15 22:37:00 +00:00
(testing (str "redirects on POST not followed by persistent Java client "
"when forceRedirects option not set to true")
(let [client (Async/createClient (ClientOptions.))]
(try
(let [request-options (RequestOptions.
(URI. "http://localhost:8080/hello"))
response (.post client request-options)]
(is (= 302 (.getStatus (.deref response)))))
(finally
(.close client)))))
(testing "redirects on POST followed by Java client when option is set"
(let [client (Async/createClient (.. (ClientOptions.)
(setForceRedirects true)))]
(try
(let [request-options (RequestOptions.
(URI. "http://localhost:8080/hello"))
response (.post client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= "Hello, World!" (slurp (.getBody (.deref response))))))
(finally
(.close client)))))
(testing "redirects not followed by Java client when :follow-redirects is false"
(let [client (Async/createClient (.. (ClientOptions.)
(setFollowRedirects false)))]
(try
(let [request-options (RequestOptions.
(URI. "http://localhost:8080/hello"))
response (.get client request-options)]
(is (= 302 (.getStatus (.deref response)))))
(finally
(.close client)))))
(testing ":follow-redirects overrides :force-redirects for Java client"
(let [client (Async/createClient (.. (ClientOptions.)
(setFollowRedirects false)
(setForceRedirects true)))]
(try
(let [request-options (RequestOptions.
(URI. "http://localhost:8080/hello"))
response (.get client request-options)]
(is (= 302 (.getStatus (.deref response)))))
(finally
(.close client)))))
(testing (str "redirects on POST not followed by clojure client "
"when :force-redirects is not set to true")
(with-open [client (async/create-client {:force-redirects false})]
(let [opts {:method :post
:url "http://localhost:8080/hello"
:as :text}
response (common/post client "http://localhost:8080/hello" opts)]
(is (= 302 (:status @response))))))
(PE-5623) Add :force-redirects to clj client Add a :force-redirects option to the clojure client that will relax the redirect restrictions put on the client for a POST or PUT method.
2014-08-25 18:15:22 +00:00
(testing (str "redirects on POST followed by persistent clojure client "
"when option is set")
(TK-103) Restored redirect-test-async tests This commit restores several of the `rediret-test-async` tests, using a client to make the requests since the client-less single-request async APIs are no longer available.
2014-11-15 22:37:00 +00:00
(with-open [client (async/create-client {:force-redirects true})]
(let [response (common/post client
"http://localhost:8080/hello"
{:as :text})]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (:body @response))))))
(PE-5623) Add :follow-redirects option clj clients Add a :follow-redirects option to the clj clients which, when set to false, will prevent the client from following a redirect.
2014-08-26 23:36:47 +00:00
(testing (str "persistent clojure client does not follow redirects when "
":follow-redirects is set to false")
(TK-103) Restored redirect-test-async tests This commit restores several of the `rediret-test-async` tests, using a client to make the requests since the client-less single-request async APIs are no longer available.
2014-11-15 22:37:00 +00:00
(with-open [client (async/create-client {:follow-redirects false})]
(let [response (common/get client
"http://localhost:8080/hello"
{:as :text})]
(is (= 302 (:status @response))))))
(PE-5623) Add :follow-redirects option clj clients Add a :follow-redirects option to the clj clients which, when set to false, will prevent the client from following a redirect.
2014-08-26 23:36:47 +00:00
(testing ":follow-redirects overrides :force-redirects with persistent clj client"
(TK-103) Restored redirect-test-async tests This commit restores several of the `rediret-test-async` tests, using a client to make the requests since the client-less single-request async APIs are no longer available.
2014-11-15 22:37:00 +00:00
(with-open [client (async/create-client {:follow-redirects false
:force-redirects true})]
(let [response (common/get client
"http://localhost:8080/hello"
{:as :text})]
(is (= 302 (:status @response)))))))))
(TK-134) Add connect and socket timeout client configuration This commit enables http clients to optionally configure a connect and/or socket timeout for requests. For persistent clients, the timeout values can only be configured at the client and not a per-request level. Non-persistent client requests support the configuration of these timeouts. This commit also bumps a few dependencies - tk-jetty9 to 1.2.0, trapperkeeper to 1.10, ssl-utils to 0.8.0, and clojure to 1.6.0 - and adds the 'test' tk-jetty9 library as a dev dependency for testing.
2015-03-16 17:50:09 +00:00
(deftest short-connect-timeout-persistent-java-test-async
(testing (str "connection times out properly for java persistent client "
"async request with short timeout")
(with-open [client (-> (ClientOptions.)
(.setConnectTimeoutMilliseconds 250)
(Async/createClient))]
(let [request-options (RequestOptions. "http://127.0.0.255:65535")
time-before-connect (System/currentTimeMillis)]
(TK-134) Updated connection tests to handle Connect or SocketTimeout This commit updates the short connect timeout tests to tolerate either a ConnectException or SocketTimeoutException as a success condition. Connections could fail with either exception depending upon timing or network setup.
2015-03-17 05:46:07 +00:00
(is (connect-exception-thrown? (-> client
(.get request-options)
(.deref)
(.getError)))
(TK-134) Add connect and socket timeout client configuration This commit enables http clients to optionally configure a connect and/or socket timeout for requests. For persistent clients, the timeout values can only be configured at the client and not a per-request level. Non-persistent client requests support the configuration of these timeouts. This commit also bumps a few dependencies - tk-jetty9 to 1.2.0, trapperkeeper to 1.10, ssl-utils to 0.8.0, and clojure to 1.6.0 - and adds the 'test' tk-jetty9 library as a dev dependency for testing.
2015-03-16 17:50:09 +00:00
"Unexpected result for connection attempt")
(is (elapsed-within-range? time-before-connect 2000)
"Connection attempt took significantly longer than timeout")))))
(deftest short-connect-timeout-persistent-clojure-test-async
(testing (str "connection times out properly for clojure persistent client "
"async request with short timeout")
(with-open [client (async/create-client
{:connect-timeout-milliseconds 250})]
(let [time-before-connect (System/currentTimeMillis)]
(TK-134) Updated connection tests to handle Connect or SocketTimeout This commit updates the short connect timeout tests to tolerate either a ConnectException or SocketTimeoutException as a success condition. Connections could fail with either exception depending upon timing or network setup.
2015-03-17 05:46:07 +00:00
(is (connect-exception-thrown? (-> @(common/get
client
"http://127.0.0.255:65535")
:error))
(TK-134) Add connect and socket timeout client configuration This commit enables http clients to optionally configure a connect and/or socket timeout for requests. For persistent clients, the timeout values can only be configured at the client and not a per-request level. Non-persistent client requests support the configuration of these timeouts. This commit also bumps a few dependencies - tk-jetty9 to 1.2.0, trapperkeeper to 1.10, ssl-utils to 0.8.0, and clojure to 1.6.0 - and adds the 'test' tk-jetty9 library as a dev dependency for testing.
2015-03-16 17:50:09 +00:00
"Unexpected result for connection attempt")
(is (elapsed-within-range? time-before-connect 2000)
"Connection attempt took significantly longer than timeout")))))
(deftest longer-connect-timeout-test-async
(testing "connection succeeds for async request with longer connect timeout"
(testlogging/with-test-logging
(testwebserver/with-test-webserver app port
(let [url (str "http://localhost:" port "/hello")]
(testing "java persistent async client"
(with-open [client (-> (ClientOptions.)
(.setConnectTimeoutMilliseconds 2000)
(Async/createClient))]
(let [response (-> client
(.get (RequestOptions. url))
(.deref))]
(is (= 200 (.getStatus response)))
(is (= "Hello, World!" (slurp (.getBody response)))))))
(testing "clojure persistent async client"
(with-open [client (async/create-client
{:connect-timeout-milliseconds 2000})]
(let [response @(common/get client url {:as :text})]
(is (= 200 (:status response)))
(is (= "Hello, World!" (:body response)))))))))))
(deftest short-socket-timeout-persistent-java-test-async
(testing (str "socket read times out properly for persistent java async "
"request with short timeout")
(with-open [client (-> (ClientOptions.)
(.setSocketTimeoutMilliseconds 1)
(Async/createClient))
server (ServerSocket. 0)]
(let [request-options (-> "http://127.0.0.1:"
(str (.getLocalPort server))
(RequestOptions.))
time-before-connect (System/currentTimeMillis)]
(is (instance? SocketTimeoutException (-> client
(.get request-options)
(.deref)
(.getError)))
"Unexpected result for get attempt")
(is (elapsed-within-range? time-before-connect 2000)
"Get attempt took significantly longer than timeout")))))
(MAINT) Fix timeout test names and testing strings This commit has a couple of cosmetic fixes to the names and testing strings for a few tests introduced by TK-134.
2015-03-20 04:25:55 +00:00
(deftest short-socket-timeout-persistent-clojure-test-async
(TK-134) Add connect and socket timeout client configuration This commit enables http clients to optionally configure a connect and/or socket timeout for requests. For persistent clients, the timeout values can only be configured at the client and not a per-request level. Non-persistent client requests support the configuration of these timeouts. This commit also bumps a few dependencies - tk-jetty9 to 1.2.0, trapperkeeper to 1.10, ssl-utils to 0.8.0, and clojure to 1.6.0 - and adds the 'test' tk-jetty9 library as a dev dependency for testing.
2015-03-16 17:50:09 +00:00
(testing (str "socket read times out properly for clojure persistent client "
"async request with short timeout")
(with-open [client (async/create-client
{:socket-timeout-milliseconds 250})
server (ServerSocket. 0)]
(let [url (str "http://127.0.0.1:" (.getLocalPort server))
time-before-connect (System/currentTimeMillis)]
(is (instance? SocketTimeoutException
(-> @(common/get client url)
:error))
"Unexpected result for get attempt")
(is (elapsed-within-range? time-before-connect 2000)
"Get attempt took significantly longer than timeout")))))
(deftest longer-socket-timeout-test-async
(MAINT) Fix timeout test names and testing strings This commit has a couple of cosmetic fixes to the names and testing strings for a few tests introduced by TK-134.
2015-03-20 04:25:55 +00:00
(testing "get succeeds for async request with longer socket timeout"
(TK-134) Add connect and socket timeout client configuration This commit enables http clients to optionally configure a connect and/or socket timeout for requests. For persistent clients, the timeout values can only be configured at the client and not a per-request level. Non-persistent client requests support the configuration of these timeouts. This commit also bumps a few dependencies - tk-jetty9 to 1.2.0, trapperkeeper to 1.10, ssl-utils to 0.8.0, and clojure to 1.6.0 - and adds the 'test' tk-jetty9 library as a dev dependency for testing.
2015-03-16 17:50:09 +00:00
(testlogging/with-test-logging
(testwebserver/with-test-webserver app port
(let [url (str "http://localhost:" port "/hello")]
(testing "java persistent async client"
(with-open [client (-> (ClientOptions.)
(.setSocketTimeoutMilliseconds 2000)
(Async/createClient))]
(let [response (-> client
(.get (RequestOptions. url))
(.deref))]
(is (= 200 (.getStatus response)))
(is (= "Hello, World!" (slurp (.getBody response)))))))
(testing "clojure persistent async client"
(with-open [client (async/create-client
{:socket-timeout-milliseconds 2000})]
(let [response @(common/get client url {:as :text})]
(is (= 200 (:status response)))
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
(is (= "Hello, World!" (:body response)))))))))))
Rework tests for improved clarity and AsyncConsumer releaseResources() fix
2015-10-13 10:40:32 +00:00
(defn- generate-data
"Generate data of approximately the requested size, which is moderately compressible"
[data-size]
(apply str "xxxx" (repeatedly (/ data-size 35) #(UUID/randomUUID))))
(defn- successful-handler
"A Ring handler that asynchronously sends some data, waits for confirmation the data has been received then sends
some more data"
[data send-more-data]
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
(fn [_]
(let [outstream (PipedOutputStream.)
Rework tests for improved clarity and AsyncConsumer releaseResources() fix
2015-10-13 10:40:32 +00:00
instream (PipedInputStream.)]
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
(.connect instream outstream)
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
;; Return the response immediately and asynchronously stream some data into it
(future
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
(.write outstream (.getBytes data))
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
; Block until the client confirms it has read the first few bytes
Rework tests for improved clarity and AsyncConsumer releaseResources() fix
2015-10-13 10:40:32 +00:00
; :socket-timeout-milliseconds on the client ensures we can't really get stuck here, even if the test fails
(if send-more-data (deref send-more-data))
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
; Write the last of the data
Rework tests for improved clarity and AsyncConsumer releaseResources() fix
2015-10-13 10:40:32 +00:00
(.write outstream (.getBytes "yyyy"))
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
(.close outstream))
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
{:status 200
:body instream})))
Rework tests for improved clarity and AsyncConsumer releaseResources() fix
2015-10-13 10:40:32 +00:00
(defn- blocking-handler
"A Ring handler that sends some data but then never closes the socket"
[data]
(fn [_]
(let [outstream (PipedOutputStream.)
instream (PipedInputStream.)]
(.connect instream outstream)
;; Return the response immediately and asynchronously stream some data into it
(future
(.write outstream (.getBytes data)))
{:status 200
:body instream})))
(defn- clojure-non-blocking-streaming
"Stream 32M of data (roughly) which is large enough to ensure the client won't buffer it all. Checks the data is
streamed in a non-blocking manner i.e some data is received by the client before the server has finished
transmission"
[decompress-body?]
(testlogging/with-test-logging
(let [data (generate-data (* 32 1024 1024))
opts {:as :unbuffered-stream :decompress-body decompress-body?}]
(testing " - check data can be streamed successfully success"
(let [send-more-data (promise)]
(testwebserver/with-test-webserver-and-config
(successful-handler data send-more-data) port {:shutdown-timeout-seconds 1}
(with-open [client (async/create-client {:connect-timeout-milliseconds 100
:socket-timeout-milliseconds 20000})]
(let [response @(common/get client (str "http://localhost:" port "/hello") opts)
{:keys [status body]} response]
(is (= 200 status))
(let [instream body
buf (make-array Byte/TYPE 4)
_ (.read instream buf)]
(is (= "xxxx" (String. buf "UTF-8"))) ;; Make sure we can read a few chars off of the stream
(deliver send-more-data true) ;; Indicate we read some chars
(is (= (str data "yyyy") (str "xxxx" (slurp instream)))))))))) ;; Read the rest and validate
(testing " - check socket timeout is handled"
(try
(testwebserver/with-test-webserver-and-config
(blocking-handler data) port {:shutdown-timeout-seconds 1}
(with-open [client (async/create-client {:connect-timeout-milliseconds 100
:socket-timeout-milliseconds 200})]
(let [response @(common/get client (str "http://localhost:" port "/hello") opts)
{:keys [body error]} response]
(is (nil? error))
;; Consume the body to get the exception
(is (thrown? SocketTimeoutException (slurp body))))))
(catch TimeoutException e
;; Expected whenever a server-side failure is generated
)))
(testing " - check connection timeout is handled"
(with-open [client (async/create-client {:connect-timeout-milliseconds 100})]
(let [response @(common/get client (str "http://localhost:" 12345 "/bad") opts)
{:keys [error]} response]
(is error)
(is (instance? ConnectException error))))))))
(deftest clojure-non-blocking-streaming-without-decompression
(testing "clojure :unbuffered-stream with 32MB payload and no decompression"
(clojure-non-blocking-streaming false)))
(deftest clojure-non-blocking-streaming-with-decompression
(testing "clojure :unbuffered-stream with 32MB payload and decompression"
(clojure-non-blocking-streaming true)))
(defn- clojure-blocking-streaming
"Stream data that is buffered client-side i.e. in a blocking manner"
[data opts]
(testlogging/with-test-logging
(testing " - check data can be streamed successfully success"
(testwebserver/with-test-webserver-and-config
(successful-handler data nil) port {:shutdown-timeout-seconds 1}
(with-open [client (async/create-client {:connect-timeout-milliseconds 100
:socket-timeout-milliseconds 20000})]
(let [response @(common/get client (str "http://localhost:" port "/hello") opts)
{:keys [status body]} response]
(is (= 200 status))
(let [instream body
buf (make-array Byte/TYPE 4)
_ (.read instream buf)]
(is (= "xxxx" (String. buf "UTF-8"))) ;; Make sure we can read a few chars off of the stream
(is (= (str data "yyyy") (str "xxxx" (slurp instream))))))))) ;; Read the rest and validate
(testing " - check socket timeout is handled"
(try
(testwebserver/with-test-webserver-and-config
(blocking-handler data) port {:shutdown-timeout-seconds 1}
(with-open [client (async/create-client {:connect-timeout-milliseconds 100
:socket-timeout-milliseconds 200})]
(let [response @(common/get client (str "http://localhost:" port "/hello") opts)
{:keys [error]} response]
(is (instance? SocketTimeoutException error)))))
(catch TimeoutException e
;; Expected whenever a server-side failure is generated
)))
(testing " - check connection timeout is handled"
(with-open [client (async/create-client {:connect-timeout-milliseconds 100})]
(let [response @(common/get client (str "http://localhost:" 12345 "/bad") opts)
{:keys [error]} response]
(is error)
(is (instance? ConnectException error)))))))
(deftest clojure-blocking-streaming-without-decompression
(testing "clojure :unbuffered-stream with 1K payload and no decompression"
;; This is a small enough payload that :unbuffered-stream still buffered it all in memory and so it behaves
;; identically to :stream
(clojure-blocking-streaming (generate-data 1024) {:as :unbuffered-stream :decompress-body false})))
(deftest clojure-blocking-streaming-with-decompression
(testing "clojure :unbuffered-stream with 1K payload and decompression"
;; This is a small enough payload that :unbuffered-stream still buffered it all in memory and so it behaves
;; identically to :stream
(clojure-blocking-streaming (generate-data 1024) {:as :unbuffered-stream :decompress-body true})))
(deftest clojure-existing-streaming-with-small-payload-without-decompression
(testing "clojure :stream with 1K payload and no decompression"
(clojure-blocking-streaming (generate-data 1024) {:as :stream :decompress-body false})))
(deftest clojure-existing-streaming-with-small-payload-with-decompression
(testing "clojure :stream with 1K payload and decompression"
(clojure-blocking-streaming (generate-data 1024) {:as :stream :decompress-body true})))
(deftest clojure-existing-streaming-with-large-payload-without-decompression
(testing "clojure :stream with 32M payload and no decompression"
(clojure-blocking-streaming (generate-data (* 32 1024 1024)) {:as :stream :decompress-body false})))
(deftest clojure-existing-streaming-with-large-payload-with-decompression
(testing "clojure :stream with 32M payload and decompression"
(clojure-blocking-streaming (generate-data (* 32 1024 1024)) {:as :stream :decompress-body true})))
Reference in a new issue Copy permalink