GNU Parallel

This command would request a bunch of web addresses in parallel. It would also pause if the current system load is over 10.

time cat urls \
  | parallel --load 10 -q \
    curl -s -q -w %{url_effective}:%{http_code}:%{time_total}'\n' {}  -o /dev/null \
  | tee curl_log