yogsototh/clj-http-client
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
clj-http-client/test/puppetlabs/http/client/async_plaintext_test.clj

466 lines
24 KiB
Clojure
Raw Normal View History

(TK-23) Port to apache HttpAsyncClient * Ports the code over to use the Apache HttpAsyncClient library instead of http-kit, as we were getting some weird and inconsistent SSL errors from http-kit, and * Simplify the API (by eliminating a lot of the supported configuration options). Some of these options we'll eventually want to add back in, but for now, getting rid of them makes the API less cluttered and also gives us the opportunity to do a better job writing tests for new options when we do add support for them. For now it only supports constructing a new client on every request. I intend to add API for creating a client explicitly, and a protocol for how to interact with the client. This will allow users to re-use an existing client as they see fit... but this hasn't been implemented yet.
2014-07-03 15:58:46 +00:00
(ns puppetlabs.http.client.async-plaintext-test
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
(:import (com.puppetlabs.http.client Async RequestOptions ClientOptions ResponseBodyType)
(TK-29) Move to URI instead of String Have the RequestOptions constructor take a URI instead of a string. Add signature to the HTTP request methods that take a URI.
2014-08-08 23:32:48 +00:00
(org.apache.http.impl.nio.client HttpAsyncClients)
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
(java.net URI SocketTimeoutException ServerSocket ConnectException)
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
(java.io PipedInputStream PipedOutputStream)
(java.util.concurrent TimeoutException))
(TK-23) Port to apache HttpAsyncClient * Ports the code over to use the Apache HttpAsyncClient library instead of http-kit, as we were getting some weird and inconsistent SSL errors from http-kit, and * Simplify the API (by eliminating a lot of the supported configuration options). Some of these options we'll eventually want to add back in, but for now, getting rid of them makes the API less cluttered and also gives us the opportunity to do a better job writing tests for new options when we do add support for them. For now it only supports constructing a new client on every request. I intend to add API for creating a client explicitly, and a protocol for how to interact with the client. This will allow users to re-use an existing client as they see fit... but this hasn't been implemented yet.
2014-07-03 15:58:46 +00:00
(:require [clojure.test :refer :all]
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
[clojure.java.io :as io]
(TK-29) Add query params to Java client Add query params back into the Java client, along with tests to validate this functionality is working.
2014-08-06 17:49:48 +00:00
[puppetlabs.http.client.test-common :refer :all]
(TK-23) Port to apache HttpAsyncClient * Ports the code over to use the Apache HttpAsyncClient library instead of http-kit, as we were getting some weird and inconsistent SSL errors from http-kit, and * Simplify the API (by eliminating a lot of the supported configuration options). Some of these options we'll eventually want to add back in, but for now, getting rid of them makes the API less cluttered and also gives us the opportunity to do a better job writing tests for new options when we do add support for them. For now it only supports constructing a new client on every request. I intend to add API for creating a client explicitly, and a protocol for how to interact with the client. This will allow users to re-use an existing client as they see fit... but this hasn't been implemented yet.
2014-07-03 15:58:46 +00:00
[puppetlabs.trapperkeeper.core :as tk]
[puppetlabs.trapperkeeper.testutils.bootstrap :as testutils]
[puppetlabs.trapperkeeper.testutils.logging :as testlogging]
(TK-134) Add connect and socket timeout client configuration This commit enables http clients to optionally configure a connect and/or socket timeout for requests. For persistent clients, the timeout values can only be configured at the client and not a per-request level. Non-persistent client requests support the configuration of these timeouts. This commit also bumps a few dependencies - tk-jetty9 to 1.2.0, trapperkeeper to 1.10, ssl-utils to 0.8.0, and clojure to 1.6.0 - and adds the 'test' tk-jetty9 library as a dev dependency for testing.
2015-03-16 17:50:09 +00:00
[puppetlabs.trapperkeeper.testutils.webserver :as testwebserver]
(TK-23) Port to apache HttpAsyncClient * Ports the code over to use the Apache HttpAsyncClient library instead of http-kit, as we were getting some weird and inconsistent SSL errors from http-kit, and * Simplify the API (by eliminating a lot of the supported configuration options). Some of these options we'll eventually want to add back in, but for now, getting rid of them makes the API less cluttered and also gives us the opportunity to do a better job writing tests for new options when we do add support for them. For now it only supports constructing a new client on every request. I intend to add API for creating a client explicitly, and a protocol for how to interact with the client. This will allow users to re-use an existing client as they see fit... but this hasn't been implemented yet.
2014-07-03 15:58:46 +00:00
[puppetlabs.trapperkeeper.services.webserver.jetty9-service :as jetty9]
(TK-27) Refactor client protocols Refactor the async and sync client protocols into a single protocol which resides in a new file, common.clj.
2014-07-15 23:46:20 +00:00
[puppetlabs.http.client.common :as common]
(TK-23) Port to apache HttpAsyncClient * Ports the code over to use the Apache HttpAsyncClient library instead of http-kit, as we were getting some weird and inconsistent SSL errors from http-kit, and * Simplify the API (by eliminating a lot of the supported configuration options). Some of these options we'll eventually want to add back in, but for now, getting rid of them makes the API less cluttered and also gives us the opportunity to do a better job writing tests for new options when we do add support for them. For now it only supports constructing a new client on every request. I intend to add API for creating a client explicitly, and a protocol for how to interact with the client. This will allow users to re-use an existing client as they see fit... but this hasn't been implemented yet.
2014-07-03 15:58:46 +00:00
[puppetlabs.http.client.async :as async]
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
[schema.test :as schema-test]
[clojure.tools.logging :as log]))
(TK-23) Port to apache HttpAsyncClient * Ports the code over to use the Apache HttpAsyncClient library instead of http-kit, as we were getting some weird and inconsistent SSL errors from http-kit, and * Simplify the API (by eliminating a lot of the supported configuration options). Some of these options we'll eventually want to add back in, but for now, getting rid of them makes the API less cluttered and also gives us the opportunity to do a better job writing tests for new options when we do add support for them. For now it only supports constructing a new client on every request. I intend to add API for creating a client explicitly, and a protocol for how to interact with the client. This will allow users to re-use an existing client as they see fit... but this hasn't been implemented yet.
2014-07-03 15:58:46 +00:00
(use-fixtures :once schema-test/validate-schemas)
(defn app
(TK-134) Add connect and socket timeout client configuration This commit enables http clients to optionally configure a connect and/or socket timeout for requests. For persistent clients, the timeout values can only be configured at the client and not a per-request level. Non-persistent client requests support the configuration of these timeouts. This commit also bumps a few dependencies - tk-jetty9 to 1.2.0, trapperkeeper to 1.10, ssl-utils to 0.8.0, and clojure to 1.6.0 - and adds the 'test' tk-jetty9 library as a dev dependency for testing.
2015-03-16 17:50:09 +00:00
[_]
(TK-23) Port to apache HttpAsyncClient * Ports the code over to use the Apache HttpAsyncClient library instead of http-kit, as we were getting some weird and inconsistent SSL errors from http-kit, and * Simplify the API (by eliminating a lot of the supported configuration options). Some of these options we'll eventually want to add back in, but for now, getting rid of them makes the API less cluttered and also gives us the opportunity to do a better job writing tests for new options when we do add support for them. For now it only supports constructing a new client on every request. I intend to add API for creating a client explicitly, and a protocol for how to interact with the client. This will allow users to re-use an existing client as they see fit... but this hasn't been implemented yet.
2014-07-03 15:58:46 +00:00
{:status 200
:body "Hello, World!"})
(tk/defservice test-web-service
[[:WebserverService add-ring-handler]]
(init [this context]
(add-ring-handler app "/hello")
context))
(TK-27) Move test Move persistent async client test back into the async-plaintext-test file.
2014-07-15 22:07:43 +00:00
(deftest persistent-async-client-test
(testlogging/with-test-logging
(testutils/with-app-with-config app
[jetty9/jetty9-service test-web-service]
{:webserver {:port 10000}}
(TK-108) Add a persistent async Java client Add a new class, PersistentAsyncHttpClient, that is a wrapped CloseableHttpAsyncClient that is persistent.
2014-11-07 21:29:17 +00:00
(testing "java async client"
(let [request-options (RequestOptions. (URI. "http://localhost:10000/hello/"))
client-options (ClientOptions.)
(TK-108) Return interface in createClient methods Return interfaces in the createClient methods rather than the implementation classes themselves.
2014-11-10 19:24:36 +00:00
client (Async/createClient client-options)]
(TK-108) Add a persistent async Java client Add a new class, PersistentAsyncHttpClient, that is a wrapped CloseableHttpAsyncClient that is persistent.
2014-11-07 21:29:17 +00:00
(testing "HEAD request with persistent async client"
(let [response (.head client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= nil (.getBody (.deref response))))))
(testing "GET request with persistent async client"
(let [response (.get client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= "Hello, World!" (slurp (.getBody (.deref response)))))))
(testing "POST request with persistent async client"
(let [response (.post client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= "Hello, World!" (slurp (.getBody (.deref response)))))))
(testing "PUT request with persistent async client"
(let [response (.put client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= "Hello, World!" (slurp (.getBody (.deref response)))))))
(testing "DELETE request with persistent async client"
(let [response (.delete client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= "Hello, World!" (slurp (.getBody (.deref response)))))))
(testing "TRACE request with persistent async client"
(let [response (.trace client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= "Hello, World!" (slurp (.getBody (.deref response)))))))
(testing "OPTIONS request with persistent async client"
(let [response (.options client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= "Hello, World!" (slurp (.getBody (.deref response)))))))
(testing "PATCH request with persistent async client"
(let [response (.patch client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= "Hello, World!" (slurp (.getBody (.deref response)))))))
(TK-103) Restore client close tests This commit restores some of the client close tests which had been disabled temporarily per some of the refactoring for TK-108.
2014-11-12 22:20:50 +00:00
(testing "client closes properly"
(.close client)
(is (thrown? IllegalStateException
(.get client request-options))))))
(TK-108) Add a persistent async Java client Add a new class, PersistentAsyncHttpClient, that is a wrapped CloseableHttpAsyncClient that is persistent.
2014-11-07 21:29:17 +00:00
(testing "clojure async client"
(let [client (async/create-client {})]
(testing "HEAD request with persistent async client"
(let [response (common/head client "http://localhost:10000/hello/")]
(is (= 200 (:status @response)))
(is (= nil (:body @response)))))
(testing "GET request with persistent async client"
(let [response (common/get client "http://localhost:10000/hello/")]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(testing "POST request with persistent async client"
(let [response (common/post client "http://localhost:10000/hello/")]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(testing "PUT request with persistent async client"
(let [response (common/put client "http://localhost:10000/hello/")]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(testing "DELETE request with persistent async client"
(let [response (common/delete client "http://localhost:10000/hello/")]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(testing "TRACE request with persistent async client"
(let [response (common/trace client "http://localhost:10000/hello/")]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(testing "OPTIONS request with persistent async client"
(let [response (common/options client "http://localhost:10000/hello/")]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(testing "PATCH request with persistent async client"
(let [response (common/patch client "http://localhost:10000/hello/")]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(testing "client closes properly"
(common/close client)
(TK-103) Restore client close tests This commit restores some of the client close tests which had been disabled temporarily per some of the refactoring for TK-108.
2014-11-12 22:20:50 +00:00
(is (thrown? IllegalStateException
(common/get client
"http://localhost:10000/hello/")))))))))
(TK-27) Add tests for request-with-client Add asynchronous and synchronous tests for request-with-client.
2014-07-17 21:03:39 +00:00
(deftest request-with-client-test
(testlogging/with-test-logging
(testutils/with-app-with-config app
[jetty9/jetty9-service test-web-service]
{:webserver {:port 10000}}
(let [client (HttpAsyncClients/createDefault)
opts {:method :get :url "http://localhost:10000/hello/"}]
(.start client)
(testing "GET request works with request-with-client"
(let [response (async/request-with-client opts nil client)]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(testing "Client persists when passed to request-with-client"
(let [response (async/request-with-client opts nil client)]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (slurp (:body @response))))))
(TK-29) Add query params to Java client Add query params back into the Java client, along with tests to validate this functionality is working.
2014-08-06 17:49:48 +00:00
(.close client)))))
(TK-103) Restored query-params-test-async tests This commit restores the `query-params-test-async` tests, using a client to make the requests since the client-less single-request async APIs are no longer available.
2014-11-14 23:36:40 +00:00
(deftest query-params-test-async
(testlogging/with-test-logging
(testutils/with-app-with-config app
[jetty9/jetty9-service test-params-web-service]
{:webserver {:port 8080}}
(testing "URL Query Parameters work with the Java client"
(let [client (Async/createClient (ClientOptions.))]
(try
(let [request-options (RequestOptions.
(URI. "http://localhost:8080/params?foo=bar&baz=lux"))
response (.get client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= queryparams (read-string (slurp (.getBody
(.deref response)))))))
(finally
(.close client)))))
(testing "URL Query Parameters work with the clojure client"
(with-open [client (async/create-client {})]
(let [opts {:method :get
:url "http://localhost:8080/params/"
:query-params queryparams
:as :text}
response (common/get client "http://localhost:8080/params" opts)]
(is (= 200 (:status @response)))
(is (= queryparams (read-string (:body @response)))))))
(testing "URL Query Parameters can be set directly in the URL"
(with-open [client (async/create-client {})]
(let [response (common/get client
"http://localhost:8080/params?paramone=one"
{:as :text})]
(is (= 200 (:status @response)))
(is (= (str {"paramone" "one"}) (:body @response))))))
(testing (str "URL Query Parameters set in URL are overwritten if params "
"are also specified in options map")
(with-open [client (async/create-client {})]
(let [response (common/get client
"http://localhost:8080/params?paramone=one&foo=lux"
query-options)]
(is (= 200 (:status @response)))
(is (= queryparams (read-string (:body @response))))))))))
(PE-5623) Add :force-redirects to clj client Add a :force-redirects option to the clojure client that will relax the redirect restrictions put on the client for a POST or PUT method.
2014-08-25 18:15:22 +00:00
(deftest redirect-test-async
(testlogging/with-test-logging
(testutils/with-app-with-config app
[jetty9/jetty9-service redirect-web-service]
{:webserver {:port 8080}}
(TK-103) Restored redirect-test-async tests This commit restores several of the `rediret-test-async` tests, using a client to make the requests since the client-less single-request async APIs are no longer available.
2014-11-15 22:37:00 +00:00
(testing (str "redirects on POST not followed by persistent Java client "
"when forceRedirects option not set to true")
(let [client (Async/createClient (ClientOptions.))]
(try
(let [request-options (RequestOptions.
(URI. "http://localhost:8080/hello"))
response (.post client request-options)]
(is (= 302 (.getStatus (.deref response)))))
(finally
(.close client)))))
(testing "redirects on POST followed by Java client when option is set"
(let [client (Async/createClient (.. (ClientOptions.)
(setForceRedirects true)))]
(try
(let [request-options (RequestOptions.
(URI. "http://localhost:8080/hello"))
response (.post client request-options)]
(is (= 200 (.getStatus (.deref response))))
(is (= "Hello, World!" (slurp (.getBody (.deref response))))))
(finally
(.close client)))))
(testing "redirects not followed by Java client when :follow-redirects is false"
(let [client (Async/createClient (.. (ClientOptions.)
(setFollowRedirects false)))]
(try
(let [request-options (RequestOptions.
(URI. "http://localhost:8080/hello"))
response (.get client request-options)]
(is (= 302 (.getStatus (.deref response)))))
(finally
(.close client)))))
(testing ":follow-redirects overrides :force-redirects for Java client"
(let [client (Async/createClient (.. (ClientOptions.)
(setFollowRedirects false)
(setForceRedirects true)))]
(try
(let [request-options (RequestOptions.
(URI. "http://localhost:8080/hello"))
response (.get client request-options)]
(is (= 302 (.getStatus (.deref response)))))
(finally
(.close client)))))
(testing (str "redirects on POST not followed by clojure client "
"when :force-redirects is not set to true")
(with-open [client (async/create-client {:force-redirects false})]
(let [opts {:method :post
:url "http://localhost:8080/hello"
:as :text}
response (common/post client "http://localhost:8080/hello" opts)]
(is (= 302 (:status @response))))))
(PE-5623) Add :force-redirects to clj client Add a :force-redirects option to the clojure client that will relax the redirect restrictions put on the client for a POST or PUT method.
2014-08-25 18:15:22 +00:00
(testing (str "redirects on POST followed by persistent clojure client "
"when option is set")
(TK-103) Restored redirect-test-async tests This commit restores several of the `rediret-test-async` tests, using a client to make the requests since the client-less single-request async APIs are no longer available.
2014-11-15 22:37:00 +00:00
(with-open [client (async/create-client {:force-redirects true})]
(let [response (common/post client
"http://localhost:8080/hello"
{:as :text})]
(is (= 200 (:status @response)))
(is (= "Hello, World!" (:body @response))))))
(PE-5623) Add :follow-redirects option clj clients Add a :follow-redirects option to the clj clients which, when set to false, will prevent the client from following a redirect.
2014-08-26 23:36:47 +00:00
(testing (str "persistent clojure client does not follow redirects when "
":follow-redirects is set to false")
(TK-103) Restored redirect-test-async tests This commit restores several of the `rediret-test-async` tests, using a client to make the requests since the client-less single-request async APIs are no longer available.
2014-11-15 22:37:00 +00:00
(with-open [client (async/create-client {:follow-redirects false})]
(let [response (common/get client
"http://localhost:8080/hello"
{:as :text})]
(is (= 302 (:status @response))))))
(PE-5623) Add :follow-redirects option clj clients Add a :follow-redirects option to the clj clients which, when set to false, will prevent the client from following a redirect.
2014-08-26 23:36:47 +00:00
(testing ":follow-redirects overrides :force-redirects with persistent clj client"
(TK-103) Restored redirect-test-async tests This commit restores several of the `rediret-test-async` tests, using a client to make the requests since the client-less single-request async APIs are no longer available.
2014-11-15 22:37:00 +00:00
(with-open [client (async/create-client {:follow-redirects false
:force-redirects true})]
(let [response (common/get client
"http://localhost:8080/hello"
{:as :text})]
(is (= 302 (:status @response)))))))))
(TK-134) Add connect and socket timeout client configuration This commit enables http clients to optionally configure a connect and/or socket timeout for requests. For persistent clients, the timeout values can only be configured at the client and not a per-request level. Non-persistent client requests support the configuration of these timeouts. This commit also bumps a few dependencies - tk-jetty9 to 1.2.0, trapperkeeper to 1.10, ssl-utils to 0.8.0, and clojure to 1.6.0 - and adds the 'test' tk-jetty9 library as a dev dependency for testing.
2015-03-16 17:50:09 +00:00
(deftest short-connect-timeout-persistent-java-test-async
(testing (str "connection times out properly for java persistent client "
"async request with short timeout")
(with-open [client (-> (ClientOptions.)
(.setConnectTimeoutMilliseconds 250)
(Async/createClient))]
(let [request-options (RequestOptions. "http://127.0.0.255:65535")
time-before-connect (System/currentTimeMillis)]
(TK-134) Updated connection tests to handle Connect or SocketTimeout This commit updates the short connect timeout tests to tolerate either a ConnectException or SocketTimeoutException as a success condition. Connections could fail with either exception depending upon timing or network setup.
2015-03-17 05:46:07 +00:00
(is (connect-exception-thrown? (-> client
(.get request-options)
(.deref)
(.getError)))
(TK-134) Add connect and socket timeout client configuration This commit enables http clients to optionally configure a connect and/or socket timeout for requests. For persistent clients, the timeout values can only be configured at the client and not a per-request level. Non-persistent client requests support the configuration of these timeouts. This commit also bumps a few dependencies - tk-jetty9 to 1.2.0, trapperkeeper to 1.10, ssl-utils to 0.8.0, and clojure to 1.6.0 - and adds the 'test' tk-jetty9 library as a dev dependency for testing.
2015-03-16 17:50:09 +00:00
"Unexpected result for connection attempt")
(is (elapsed-within-range? time-before-connect 2000)
"Connection attempt took significantly longer than timeout")))))
(deftest short-connect-timeout-persistent-clojure-test-async
(testing (str "connection times out properly for clojure persistent client "
"async request with short timeout")
(with-open [client (async/create-client
{:connect-timeout-milliseconds 250})]
(let [time-before-connect (System/currentTimeMillis)]
(TK-134) Updated connection tests to handle Connect or SocketTimeout This commit updates the short connect timeout tests to tolerate either a ConnectException or SocketTimeoutException as a success condition. Connections could fail with either exception depending upon timing or network setup.
2015-03-17 05:46:07 +00:00
(is (connect-exception-thrown? (-> @(common/get
client
"http://127.0.0.255:65535")
:error))
(TK-134) Add connect and socket timeout client configuration This commit enables http clients to optionally configure a connect and/or socket timeout for requests. For persistent clients, the timeout values can only be configured at the client and not a per-request level. Non-persistent client requests support the configuration of these timeouts. This commit also bumps a few dependencies - tk-jetty9 to 1.2.0, trapperkeeper to 1.10, ssl-utils to 0.8.0, and clojure to 1.6.0 - and adds the 'test' tk-jetty9 library as a dev dependency for testing.
2015-03-16 17:50:09 +00:00
"Unexpected result for connection attempt")
(is (elapsed-within-range? time-before-connect 2000)
"Connection attempt took significantly longer than timeout")))))
(deftest longer-connect-timeout-test-async
(testing "connection succeeds for async request with longer connect timeout"
(testlogging/with-test-logging
(testwebserver/with-test-webserver app port
(let [url (str "http://localhost:" port "/hello")]
(testing "java persistent async client"
(with-open [client (-> (ClientOptions.)
(.setConnectTimeoutMilliseconds 2000)
(Async/createClient))]
(let [response (-> client
(.get (RequestOptions. url))
(.deref))]
(is (= 200 (.getStatus response)))
(is (= "Hello, World!" (slurp (.getBody response)))))))
(testing "clojure persistent async client"
(with-open [client (async/create-client
{:connect-timeout-milliseconds 2000})]
(let [response @(common/get client url {:as :text})]
(is (= 200 (:status response)))
(is (= "Hello, World!" (:body response)))))))))))
(deftest short-socket-timeout-persistent-java-test-async
(testing (str "socket read times out properly for persistent java async "
"request with short timeout")
(with-open [client (-> (ClientOptions.)
(.setSocketTimeoutMilliseconds 1)
(Async/createClient))
server (ServerSocket. 0)]
(let [request-options (-> "http://127.0.0.1:"
(str (.getLocalPort server))
(RequestOptions.))
time-before-connect (System/currentTimeMillis)]
(is (instance? SocketTimeoutException (-> client
(.get request-options)
(.deref)
(.getError)))
"Unexpected result for get attempt")
(is (elapsed-within-range? time-before-connect 2000)
"Get attempt took significantly longer than timeout")))))
(MAINT) Fix timeout test names and testing strings This commit has a couple of cosmetic fixes to the names and testing strings for a few tests introduced by TK-134.
2015-03-20 04:25:55 +00:00
(deftest short-socket-timeout-persistent-clojure-test-async
(TK-134) Add connect and socket timeout client configuration This commit enables http clients to optionally configure a connect and/or socket timeout for requests. For persistent clients, the timeout values can only be configured at the client and not a per-request level. Non-persistent client requests support the configuration of these timeouts. This commit also bumps a few dependencies - tk-jetty9 to 1.2.0, trapperkeeper to 1.10, ssl-utils to 0.8.0, and clojure to 1.6.0 - and adds the 'test' tk-jetty9 library as a dev dependency for testing.
2015-03-16 17:50:09 +00:00
(testing (str "socket read times out properly for clojure persistent client "
"async request with short timeout")
(with-open [client (async/create-client
{:socket-timeout-milliseconds 250})
server (ServerSocket. 0)]
(let [url (str "http://127.0.0.1:" (.getLocalPort server))
time-before-connect (System/currentTimeMillis)]
(is (instance? SocketTimeoutException
(-> @(common/get client url)
:error))
"Unexpected result for get attempt")
(is (elapsed-within-range? time-before-connect 2000)
"Get attempt took significantly longer than timeout")))))
(deftest longer-socket-timeout-test-async
(MAINT) Fix timeout test names and testing strings This commit has a couple of cosmetic fixes to the names and testing strings for a few tests introduced by TK-134.
2015-03-20 04:25:55 +00:00
(testing "get succeeds for async request with longer socket timeout"
(TK-134) Add connect and socket timeout client configuration This commit enables http clients to optionally configure a connect and/or socket timeout for requests. For persistent clients, the timeout values can only be configured at the client and not a per-request level. Non-persistent client requests support the configuration of these timeouts. This commit also bumps a few dependencies - tk-jetty9 to 1.2.0, trapperkeeper to 1.10, ssl-utils to 0.8.0, and clojure to 1.6.0 - and adds the 'test' tk-jetty9 library as a dev dependency for testing.
2015-03-16 17:50:09 +00:00
(testlogging/with-test-logging
(testwebserver/with-test-webserver app port
(let [url (str "http://localhost:" port "/hello")]
(testing "java persistent async client"
(with-open [client (-> (ClientOptions.)
(.setSocketTimeoutMilliseconds 2000)
(Async/createClient))]
(let [response (-> client
(.get (RequestOptions. url))
(.deref))]
(is (= 200 (.getStatus response)))
(is (= "Hello, World!" (slurp (.getBody response)))))))
(testing "clojure persistent async client"
(with-open [client (async/create-client
{:socket-timeout-milliseconds 2000})]
(let [response @(common/get client url {:as :text})]
(is (= 200 (:status response)))
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
(is (= "Hello, World!" (:body response)))))))))))
(defn- build-content-handler
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
[data initial-bytes-read? wait-forever?]
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
(fn [_]
(let [outstream (PipedOutputStream.)
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
instream (PipedInputStream. 4)]
(.connect instream outstream)
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
;; Return the response immediately and asynchronously stream some data into it
(future
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
(.write outstream (.getBytes data))
(.flush outstream)
(if wait-forever?
; The :socket-timeout-milliseconds setting on the client means we don't actually block forever and forces
; a SocketTimeoutException on the underlying socket
(deref (promise)))
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
; Block until the client confirms it has read the first few bytes
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
; Again :socket-timeout-milliseconds on the client ensures we can't really get stuck here, even if the
; test fails
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
(if initial-bytes-read?
(deref initial-bytes-read?))
; Write the last of the data
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
(.write outstream (.getBytes "xxxx"))
(.close outstream))
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
{:status 200
:body instream})))
(defn- clojure-streaming-success
[data-size opts]
(testing (str "clojure streaming success: " data-size " bytes with opts " opts)
(let [data (apply str (repeat (- data-size 4) "x"))
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
;; If :unbuffered-stream is enabled then we check that we can read some bytes from the response before all
;; bytes have actually been transmitted
;; We need to make sure the amount of data sent is enough to ensure it doesn't get buffered by the OS or Jetty
;; About 64K seems to be the threshold
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
initial-bytes-read? (if (and (= :unbuffered-stream (:as opts))
(not (:decompress-body opts))
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
(>= data-size (* 64 1024)))
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
(promise))]
(testwebserver/with-test-webserver-and-config
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
(build-content-handler data initial-bytes-read? false) port {:shutdown-timeout-seconds 1}
(with-open [client (async/create-client {:connect-timeout-milliseconds 100 :socket-timeout-milliseconds 20000})]
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
(let [response @(common/get
client
(str "http://localhost:" port "/hello")
opts)
{:keys [status body]} response]
(is (= 200 status))
(let [instream body
buf (make-array Byte/TYPE 4)
_ (.read instream buf)]
;; Make sure we can read a few chars off of the stream
(is (= "xxxx" (String. buf "UTF-8")))
;; Indicate we read some chars
(if initial-bytes-read? (deliver initial-bytes-read? true))
;; Read the rest and validate the content
(let [final-string (str "xxxx" (slurp instream))]
(is (= (str data "xxxx") final-string))))))))))
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
(defn- clojure-streaming-socket-timeout
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
[data-size opts]
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
(testing (str "clojure streaming socket timeout: " data-size " bytes with opts " opts)
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
(let [data (apply str (repeat (- data-size 4) "x"))]
(try
(testwebserver/with-test-webserver-and-config
(build-content-handler data nil true) port {:shutdown-timeout-seconds 1}
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
(with-open [client (async/create-client {:connect-timeout-milliseconds 100 :socket-timeout-milliseconds 200})]
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
(let [response @(common/get
client
(str "http://localhost:" port "/hello")
opts)
{:keys [body error]} response]
(if (= :unbuffered-stream (:as opts))
(do
(if error
;; If there's an error this should behave the same as :stream
(is (instance? SocketTimeoutException error))
;; If there is no error, let's consume the body to get the exception
(is (thrown? SocketTimeoutException (slurp body)))))
(do
(is error)
(is (instance? SocketTimeoutException error)))))))
(catch TimeoutException e
;; Expected whenever a server-side failure is generated
)))))
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
(defn- clojure-streaming-connection-error
[opts]
(testing (str "clojure streaming connect error")
(with-open [client (async/create-client {:connect-timeout-milliseconds 100})]
(let [response @(common/get
client
(str "http://localhost:" 12345 "/hello")
opts)
{:keys [error]} response]
(is error)
(is (instance? ConnectException error))))))
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
(deftest clojure-streaming
(testing "clojure streaming is consistent with different payload sizes and opts"
(testlogging/with-test-logging
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
(doseq [data-size [32 (* 1024) (* 1024 1024)]
(PE-11976) Add option to return the response early and stream the body asynchronously Adds a new :as option that uses Apache's AsyncConsumer to return the response as quickly as possible then asynchronously stream the response body using piped input/output streams.
2015-09-07 10:07:26 +00:00
decompress-body? [false true]
as [:unbuffered-stream :stream]]
(clojure-streaming-success data-size {:as as :decompress-body decompress-body?})
Incorporate comments from camlow325 - Fixes to ExceptionInsertingPipedInputStream - Fix to async consumer and removal of decompression code - Use ResponseContentEncoding to do all decompression including removing previous approach - Clarify tests and add ConnectionException
2015-10-06 12:33:55 +00:00
(clojure-streaming-socket-timeout data-size {:as as :decompress-body decompress-body?})
(clojure-streaming-connection-error {:as as :decompress-body decompress-body?})))))
Reference in a new issue Copy permalink