diff --git a/README.md b/README.md index cde0bdb..cd28817 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,24 @@ # Introduction -Measures [time-to-first-byte](https://en.wikipedia.org/wiki/Time_to_first_byte) for single or multiple URLs. Can calculate min, max & median TTFB values and log all response headers. +Measures [time-to-first-byte](https://en.wikipedia.org/wiki/Time_to_first_byte) for single or multiple URLs. Can show you quickest, slowest & median TTFB values plus optionally log all response headers. ``` Usage: ttfb [options] url [url...] - -d debug - -l (infers -d) - -n number of times to test + -d debug + -l (infers -d) log response headers. Defaults to ./curl.log + -n of times to test time to first byte + -v verbose output. Show response breakdown (DNS lookup, TLS handshake etc) ``` -Uses the calculation `%{time_starttransfer¹} - %{time_appconnect²}` which doesn't include any connection overhead, to better approximate devtool’s TTFB figure. +Implicitly follows a redirection chain using curl's `-L` option. + +Can log all response headers (the default log file is `./curl.log`) by calling with `-d`. + +Override the default log file by specifying `-l /some/file`. + +Get quickest, slowest and median TTFB values by specifying the number of times to call a URL; use `-n2` for 2 tests, `-n5` for 5 and so on. + +Uses the calculation `%{time_starttransfer¹} - %{time_appconnect²}` which doesn't include any connection overhead, to better approximate [devtool’s TTFB figure](https://developers.google.com/web/tools/chrome-devtools/network/understanding-resource-timing#slow_time_to_first_byte). ¹ [`time_starttransfer`](https://github.com/curl/curl/blob/e431daf013ea04cb1a988a2009d820224ef5fb79/docs/cmdline-opts/write-out.d#L141-L144) > The time, in seconds, it took from the start until the first byte was just about to be transferred. This includes time_pretransfer and also the time the server needed to calculate the result. @@ -19,7 +28,7 @@ Uses the calculation `%{time_starttransfer¹} - %{time_appconnect²}` which does connect/handshake to the remote host was completed. # Genesis -Based on a gist https://gist.github.com/sandeepraju/1f5fbdbdd89551ba7925abe2645f92b5 +Based on a [gist](https://gist.github.com/sandeepraju/1f5fbdbdd89551ba7925abe2645f92b5) by https://github.com/sandeepraju Modified by jay@gooby.org, [@jaygooby](https://twitter.com/jaygooby) @@ -28,24 +37,46 @@ Modified by jay@gooby.org, [@jaygooby](https://twitter.com/jaygooby) ``` Usage: ttfb [options] url [url...] - -d debug - -l (infers -d) - -n number of times to test + -d debug + -l (infers -d) log response headers. Defaults to ./curl.log + -n of times to test time to first byte + -v verbose output. Show response breakdown (DNS lookup, TLS handshake etc) ``` -Examples: +## Examples + +Basic usage: ``` $ ttfb example.com -DNS lookup: 0.523402 TLS handshake: 0.000000 TTFB including connection: 0.692724 TTFB: .692724 Total time: 0.693508 +.227436 ``` +Basic usage with verbose response breakdown: + ``` -$ ttfb -n 5 example.com +$ ttfb -v https://example.com +DNS lookup: 0.005152 TLS handshake: 0.000000 TTFB including connection: 0.200831 TTFB: .200831 Total time: 0.201132 +``` + +Test multiple times: + +``` +$ ttfb -n 5 example.com/example/url ..... fastest .177263 slowest .214302 median .179957 ``` +Test multiple URLs: + +``` +$ ttfb bbc.co.uk news.bbc.co.uk +bbc.co.uk .049985 +news.bbc.co.uk .054122 +``` + +Test multiple URLs, multiple times: + ``` $ ttfb -n 5 bbc.co.uk news.bbc.co.uk ..... @@ -54,33 +85,67 @@ bbc.co.uk fastest .030936 slowest .057755 median .034663 news.bbc.co.uk fastest .031413 slowest .182791 median .035001 ``` -``` -$ ttfb bbc.co.uk news.bbc.co.uk -bbc.co.uk DNS lookup: 0.005291 TLS handshake: 0.089403 TTFB including connection: 0.119651 TTFB: .030248 Total time: 0.506010 -news.bbc.co.uk DNS lookup: 0.004266 TLS handshake: 0.077179 TTFB including connection: 0.110649 TTFB: .033470 Total time: 0.598472 -``` - -Implicitly follows redirects using curl's `-L` - -Log all response headers (default log file is `curl.log`) by calling with `-d` - -Override the default log file by specifying `-l /some/file` - -Get min, max and median values by specifying the number of times to call -the URL (`-n2` etc) - -If you specify more than one url and have specified `-d` or `-l` the log file -will be prefixed with the URL being requested. +Verbose response breakdown when multiple tests specified: ``` -$ ttfb -d bbc.co.uk news.bbc.co.uk -bbc.co.uk DNS lookup: 0.005309 TLS handshake: 0.074216 TTFB including connection: 0.106453 TTFB: .032237 Total time: 0.462894 -news.bbc.co.uk DNS lookup: 0.005355 TLS handshake: 5.358041 TTFB including connection: 5.397377 TTFB: .039336 Total time: 6.013918 +$ ttfb -v -n 5 bbc.co.uk +DNS lookup: 0.005335 TLS handshake: 0.102314 TTFB including connection: 0.148328 TTFB: .046014 Total time: 0.646115 +DNS lookup: 0.005322 TLS handshake: 0.102609 TTFB including connection: 0.150693 TTFB: .048084 Total time: 0.644611 +DNS lookup: 0.004277 TLS handshake: 0.102066 TTFB including connection: 0.172199 TTFB: .070133 Total time: 1.196256 +DNS lookup: 0.004444 TLS handshake: 0.107375 TTFB including connection: 0.160771 TTFB: .053396 Total time: 0.637290 +DNS lookup: 0.005352 TLS handshake: 0.118882 TTFB including connection: 0.168772 TTFB: .049890 Total time: 0.653761 -$ ls -bbc_co_uk-curl.log news_bbc_co_uk-curl.log +fastest .046014 slowest .070133 median .049890 ``` +Log all the response headers for multiple tests to multiple URLs: + +``` +ttfb -d -n 2 bbc.co.uk https://www.bbc.co.uk/weather +.. +.. +bbc.co.uk fastest .027550 slowest .055215 median .041382 +https://www.bbc.co.uk/weather fastest .101020 slowest .297923 median .199471 + +$ ls *.log +bbc_co_uk-curl.log https___www_bbc_co_uk_weather-curl.log + +$ cat https___www_bbc_co_uk_weather-curl.log +HTTP/2 200 +server: openresty +x-cache-action: MISS +vary: Accept-Encoding,X-BBC-Edge-Cache,X-BBC-Edge-Scheme,X-CDN +x-cache-age: 0 +cache-control: private, stale-while-revalidate=10, max-age=0, must-revalidate +content-type: text/html;charset=utf-8 +x-mrid: w1 +date: Thu, 11 Apr 2019 17:08:07 GMT +x-xss-protection: 1; mode=block +x-content-type-options: nosniff +x-lb-nocache: true +x-msig: 24e37f81323984e4e45b8048f9e3c94a +x-frame-options: SAMEORIGIN +content-length: 1077454 + +HTTP/2 200 +server: openresty +x-cache-action: MISS +vary: Accept-Encoding,X-BBC-Edge-Cache,X-BBC-Edge-Scheme,X-CDN +x-cache-age: 0 +cache-control: private, stale-while-revalidate=10, max-age=0, must-revalidate +content-type: text/html;charset=utf-8 +x-mrid: w1 +date: Thu, 11 Apr 2019 17:08:08 GMT +x-xss-protection: 1; mode=block +x-content-type-options: nosniff +x-lb-nocache: true +x-msig: 24e37f81323984e4e45b8048f9e3c94a +x-frame-options: SAMEORIGIN +content-length: 1077454 +``` + +# More detail on time-to-first-byte + See https://blog.cloudflare.com/a-question-of-timing/ and https://curl.haxx.se/docs/manpage.html for an explanation of how the curl variables relate to the various stages of diff --git a/ttfb b/ttfb index d87ecc0..0f9e49c 100755 --- a/ttfb +++ b/ttfb @@ -1,5 +1,7 @@ #!/bin/bash # +# Shows time in seconds to first byte of a url or urls +# # Based on a gist https://gist.github.com/sandeepraju/1f5fbdbdd89551ba7925abe2645f92b5 # by https://github.com/sandeepraju # @@ -13,30 +15,33 @@ # # Examples: # -# ttfb example.com -# DNS lookup: 0.523402 TLS handshake: 0.000000 TTFB including connection: 0.692724 TTFB: .692724 Total time: 0.693508 +# ttfb https://example.com/example/url +# .098974 # -# ttfb -n 5 example.com -# min .203970 max .181486 median .190033 +# ttfb -n 5 https://example.com/ +# ..... +# fastest .099195 slowest .103138 median .099684 # # ttfb -n 5 bbc.co.uk news.bbc.co.uk -# bbc.co.uk min .032791 max .039401 median .029214 -# news.bbc.co.uk min .032927 max .032237 median .037458 +# ..... +# ..... +# bbc.co.uk fastest .045873 slowest .046870 median .045999 +# news.bbc.co.uk fastest .042286 slowest .060245 median .046035 # # ttfb bbc.co.uk news.bbc.co.uk -# bbc.co.uk DNS lookup: 0.005291 TLS handshake: 0.089403 TTFB including connection: 0.119651 TTFB: .030248 Total time: 0.506010 -# news.bbc.co.uk DNS lookup: 0.004266 TLS handshake: 0.077179 TTFB including connection: 0.110649 TTFB: .033470 Total time: 0.598472 +# bbc.co.uk .048378 +# news.bbc.co.uk .049303 # -# Implicitly follows redirects using curl's -L +# Implicitly follows redirects using curl's -L option. # # Log all response headers (default log file is ./curl.log) by calling with -d # # Override the default log file by specifying -l /some/file # # Get min, max and median values by specifying the number of times to call -# the URL (-n2 etc) +# the URL; use -n2 for 2 tests, -n5 for 5 and so on. # -# If you specify more than one url and have specified -d or -l the log file +# If you specify more than one url and have specified -d or -l, the log file # will be prefixed with the URL being requested. # # If you specify -n and -d or -l, the response headers from the consecutive @@ -55,6 +60,11 @@ # on bc and column commands. set -eu +# check dependencies +for dependency in curl bc column; do + which -s $dependency || (echo "You need to have '$dependency' installed and in your \$PATH" >&2 && exit 1) +done + # Cribbed from https://stackoverflow.com/a/41762669/391826 median() { arr=($(printf '%s\n' "${@}" | sort -n)) @@ -64,7 +74,6 @@ median() { else # Even number of elements (( j=nel/2 )) (( k=j-1 )) - # (( val=(${arr[j]} + ${arr[k]})/2 )) val=$(echo "scale=6;(${arr[j]}" + "${arr[k]})"/2|bc -l) fi echo $val @@ -83,7 +92,7 @@ do l) LOG="$OPTARG" ;; n) NUM_REQUESTS=$OPTARG ;; v) VERBOSE=1 ;; - \?) echo -e "Usage: ttfb [options] url [url...]\n\t-d debug\n\t-l (infers -d) log response headers. Defaults to curl.log\n\t-n of times to test time to first byte\n\t-v verbose output. Show request breakdown during -n calls" >&2 + \?) echo -e "Usage: ttfb [options] url [url...]\n\t-d debug\n\t-l (infers -d) log response headers. Defaults to ./curl.log\n\t-n of times to test time to first byte\n\t-v verbose output. Show response breakdown (DNS lookup, TLS handshake etc)" >&2 exit 1 ;; esac @@ -100,7 +109,7 @@ else fi # if we're given a custom log file, or log directory, implicitly set DEBUG=1 -([ -n "$LOG" ] || [ -n "$LOG_DIRECTORY" ]) && DEBUG=1 +[ -n "$LOG" ] && DEBUG=1 # default the log file to curl.log in pwd or LOG_DIRECTORY if -o was specified LOG="${LOG:-curl.log}" @@ -135,7 +144,7 @@ for URL in $URLS; do # output the url on the results line if [ ${#@} -gt 1 ]; then SHOW_URL="${URL}|" - if [ $VERBOSE -eq 1 ]; then + if [[ $VERBOSE -eq 1 && -n "$NUM_REQUESTS" && "$NUM_REQUESTS" -gt 1 ]]; then echo $URL >&2 fi else @@ -182,6 +191,10 @@ for URL in $URLS; do # show quickest, slowest and median fftb printf "${SHOW_URL}\e[32mfastest \e[39m${ttfbs[0]} \e[91mslowest \e[39m${ttfbs[${#ttfbs[*]}-1]} \e[95mmedian \e[39m$(median ${ttfbs[*]})\e[39m\n"; else - echo -e $SHOW_URL $(eval $(curl "${options[@]}" "$URL")) + if [ $VERBOSE -eq 1 ]; then + echo -e $SHOW_URL $(eval $(curl "${options[@]}" "$URL")) + else + echo -e $SHOW_URL $(eval $(curl "${options[@]}" "$URL") | grep -oE "TTFB: .{0,7}" | cut -d' ' -f2) + fi fi done | column -s'|' -t