DNSに負荷をかけるテスト
使っているDDNSサービスが腐っているので、しょうがなく評価でnode/redisでDDNSする環境を書いてみている。
APIは既存サービス互換にする必要がある。
ダメコードなのでイヤなんだけどしょうがなく合わせる。
とりあえず、DNS部分の性能がどんなものか、負荷かけてみたりしたいので、評価ツールを探す。
TCP fallback, eDNS0, DNSSECは実装してない。
DNSの負荷計測ツールって、どんなのがあるんだろう。
ググる。
フリーのDNSストレスツール の紹介 dnsperf (開発元 Nominum),dnstcpbench(開発元 Nether Labs)
http://dnsops.jp/event/20130718/20130718-stress-tool-hattori-1.pdf
DNSの評価と計測の話
https://www.nic.ad.jp/ja/materials/iw/2013/proceedings/d2/d2-hattori.pdf
queryperf
http://www.geocities.jp/yasasikukaitou/queryperf.html
だいたい、こんなとこか。
・queryperf
・dnsperf
・dnstcpbench
・resperf
dnsperf 2.0がよさげ?
CentOS7の標準パッケージにはどれもない。EPELにもない。
dnsperfは、6の時にはあったと聞くが...
Jmeter使うのかな...
queryperfはBIND関係の中にあるのかな?
$ sudo rpm -qa | grep bind-utils bind-utils-9.9.4-38.el7_3.x86_64 $ sudo rpm -ql bind-utils /etc/trusted-key.key /usr/bin/dig /usr/bin/host /usr/bin/nslookup /usr/bin/nsupdate /usr/share/man/man1/dig.1.gz /usr/share/man/man1/host.1.gz /usr/share/man/man1/nslookup.1.gz /usr/share/man/man1/nsupdate.1.gz
いないわ。
dnsperfをもらいに行く。
http://nominum.com/measurement-tools/
残念、rpmで提供されるのは6系までだった。
dnsperf and resperf 2.1.0.0 – source distributionを選んでダウンロード。
$ mkdir testtool $ cd testtool/ $ wget ftp://ftp.nominum.com/pub/nominum/dnsperf/2.1.0.0/dnsperf-src-2.1.0.0-1.tar.gz $ tar zxvf dnsperf-src-2.1.0.0-1.tar.gz dnsperf-src-2.1.0.0-1/ dnsperf-src-2.1.0.0-1/./RELEASE_NOTES dnsperf-src-2.1.0.0-1/Makefile.in dnsperf-src-2.1.0.0-1/README dnsperf-src-2.1.0.0-1/aclocal.m4 dnsperf-src-2.1.0.0-1/acx_pthread.m4 dnsperf-src-2.1.0.0-1/config.guess dnsperf-src-2.1.0.0-1/config.sub dnsperf-src-2.1.0.0-1/configure dnsperf-src-2.1.0.0-1/configure.in dnsperf-src-2.1.0.0-1/contrib/queryparse/INSTALL dnsperf-src-2.1.0.0-1/contrib/queryparse/USAGE dnsperf-src-2.1.0.0-1/contrib/queryparse/queryparse dnsperf-src-2.1.0.0-1/contrib/queryparse/queryparse.1 dnsperf-src-2.1.0.0-1/datafile.c dnsperf-src-2.1.0.0-1/datafile.h dnsperf-src-2.1.0.0-1/dns.c dnsperf-src-2.1.0.0-1/dns.h dnsperf-src-2.1.0.0-1/dnsperf.1 dnsperf-src-2.1.0.0-1/dnsperf.c dnsperf-src-2.1.0.0-1/doc/caching-dns-performance.pdf dnsperf-src-2.1.0.0-1/doc/dnsperf.pdf dnsperf-src-2.1.0.0-1/doc/resperf.pdf dnsperf-src-2.1.0.0-1/install-sh dnsperf-src-2.1.0.0-1/log.c dnsperf-src-2.1.0.0-1/log.h dnsperf-src-2.1.0.0-1/net.c dnsperf-src-2.1.0.0-1/net.h dnsperf-src-2.1.0.0-1/opt.c dnsperf-src-2.1.0.0-1/opt.h dnsperf-src-2.1.0.0-1/os.c dnsperf-src-2.1.0.0-1/os.h dnsperf-src-2.1.0.0-1/resperf-report dnsperf-src-2.1.0.0-1/resperf.1 dnsperf-src-2.1.0.0-1/resperf.c dnsperf-src-2.1.0.0-1/util.h dnsperf-src-2.1.0.0-1/version.h
README読む。
$ cat dnsperf-src-2.1.0.0-1/README
This is dnsperf, a collection of DNS server performance testing tools.
For more information, see the dnsperf(1) and resperf(1) man pages.To configure, compile, and install these programs, follow these steps.
1. Make sure that BIND 9 (9.4.0 or greater) is installed, including libraries
and header files, and that the isc-config.sh program distributed with BIND
is in your path.Note: many versions of bind do not correctly install the
header file, so if the compilation fails, obtain this file from the BIND
source distribution, and install it in the appropriate place.2. Run "sh configure" to configure the software. Most standard configure
options are supported.3. Run "make" to build dnsperf and resperf
4. Run "make install" to install dnsperf and resperf.
Additional software is available in the contrib/ directory.
BINDいるって書いてある。めんどくさいな。
とりあえず、BIND系なしでビルドして、どんなエラーになるか確認。
$ cd dnsperf-src-2.1.0.0-1/ $ ./configure checking for gcc... no checking for cc... no checking for cl.exe... no configure: error: in `/usr/local/node/ddns/testtool/dnsperf-src-2.1.0.0-1': configure: error: no acceptable C compiler found in $PATH See `config.log' for more details.
ひどいね、この環境にはgccも入ってなかったwww
$ sudo yum install gcc make (snip) Installed: gcc.x86_64 0:4.8.5-11.el7 Dependency Installed: cpp.x86_64 0:4.8.5-11.el7 glibc-devel.x86_64 0:2.17-157.el7_3.1 glibc-headers.x86_64 0:2.17-157.el7_3.1 kernel-headers.x86_64 0:3.10.0-514.2.2.el7 libmpc.x86_64 0:1.0.1-3.el7 mpfr.x86_64 0:3.1.1-4.el7 (snip)
$ ./configure checking for gcc... gcc checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking for a BSD-compatible install... /bin/install -c checking for ranlib... ranlib checking for inline... inline checking for socket in -lsocket... no checking for inet_ntoa in -lnsl... yes checking for isc-config.sh... no configure: error: BIND 9 libraries must be installed
bind-libs-9.9.4-38.el7_3.x86_64は入っているから、develか。
$ sudo yum install bind-devel (snip) Installed: bind-devel.x86_64 32:9.9.4-38.el7_3
$ ./configure checking for gcc... gcc checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking for a BSD-compatible install... /bin/install -c checking for ranlib... ranlib checking for inline... inline checking for socket in -lsocket... no checking for inet_ntoa in -lnsl... yes checking for isc-config.sh... /bin/isc-config.sh checking for socklen_t... yes checking for sa_len... no checking build system type... x86_64-unknown-linux-gnu checking host system type... x86_64-unknown-linux-gnu checking for the pthreads library -lpthreads... no checking whether pthreads work without any flags... no checking whether pthreads work with -Kthread... no checking whether pthreads work with -kthread... no checking for the pthreads library -llthread... no checking whether pthreads work with -pthread... yes checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE checking if more special flags are required for pthreads... no configure: creating ./config.status config.status: creating Makefile
これは、大勢に無関係かな。
checking for socket in -lsocket... no
checking for sa_len... no
ビルド。
$ make (snip) gcc -pthread dnsperf.o libperf.a -lnsl -L/usr/lib64 -lbind9 -ldns -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lcrypto -lisccfg -lisc -ldl -lcap -lpthread -lm -lGeoIP -L/usr/lib -lxml2 -lz -lm -ldl -lm -o dnsperf /bin/ld: cannot find -lgssapi_krb5 /bin/ld: cannot find -lkrb5 /bin/ld: cannot find -lk5crypto /bin/ld: cannot find -lcom_err /bin/ld: cannot find -lcrypto /bin/ld: cannot find -lcap collect2: error: ld returned 1 exit status make: *** [dnsperf] Error 1
libpcap-develとかlibxml2-develとかlibssl-develとかいりそうね。
$ sudo yum install libpcap-devel libxml2-devel libssl-devel (snip) Installed: libpcap-devel.x86_64 14:1.5.3-8.el7 libxml2-devel.x86_64 0:2.9.1-6.el7_2.3 Dependency Installed: xz-devel.x86_64 0:5.2.2-1.el7 zlib-devel.x86_64 0:1.2.7-17.el7
あれ?違った。krb5か。krb5-develとkrb5-libsあたりかな。
$ sudo yum install krb5-devel krb5-libs (snip) Installed: krb5-devel.x86_64 0:1.14.1-27.el7_3 Dependency Installed: keyutils-libs-devel.x86_64 0:1.5.8-3.el7 libcom_err-devel.x86_64 0:1.42.9-9.el7 libkadm5.x86_64 0:1.14.1-27.el7_3 libselinux-devel.x86_64 0:2.5-6.el7 libsepol-devel.x86_64 0:2.5-6.el7 libverto-devel.x86_64 0:0.2.5-4.el7 pcre-devel.x86_64 0:8.32-15.el7_2.1
makeする。
gcc -pthread dnsperf.o libperf.a -lnsl -L/usr/lib64 -lbind9 -ldns -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lcrypto -lisccfg -lisc -ldl -lcap -lpthread -lm -lGeoIP -L/usr/lib -lxml2 -lz -lm -ldl -lm -o dnsperf /bin/ld: cannot find -lcrypto /bin/ld: cannot find -lcap collect2: error: ld returned 1 exit status make: *** [dnsperf] Error 1
エラー減った。
このへんか。
$ sudo yum install libcap-devel (snip) Installed: libcap-devel.x86_64 0:2.22-8.el7 $ make gcc -pthread dnsperf.o libperf.a -lnsl -L/usr/lib64 -lbind9 -ldns -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lcrypto -lisccfg -lisc -ldl -lcap -lpthread -lm -lGeoIP -L/usr/lib -lxml2 -lz -lm -ldl -lm -o dnsperf /bin/ld: cannot find -lcrypto collect2: error: ld returned 1 exit status make: *** [dnsperf] Error 1
あと1つ。
openssl-libsは入っているしな。
develたりないかな。
$ sudo yum install openssl-devel (snip) Installed: openssl-devel.x86_64 1:1.0.1e-60.el7 $ make gcc -pthread dnsperf.o libperf.a -lnsl -L/usr/lib64 -lbind9 -ldns -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lcrypto -lisccfg -lisc -ldl -lcap -lpthread -lm -lGeoIP -L/usr/lib -lxml2 -lz -lm -ldl -lm -o dnsperf /bin/ld: cannot find -lGeoIP collect2: error: ld returned 1 exit status make: *** [dnsperf] Error 1
$ sudo yum install GeoIP-devel (snip) Installed: GeoIP-devel.x86_64 0:1.5.0-11.el7 $ make
通ったようだ。
# make install mkdir -p /usr/local/bin mkdir -p /usr/local/share/man/man1 /bin/install -c dnsperf /usr/local/bin /bin/install -c resperf /usr/local/bin /bin/install -c resperf-report /usr/local/bin /bin/install -c -m 644 dnsperf.1 /usr/local/share/man/man1 /bin/install -c -m 644 resperf.1 /usr/local/share/man/man1
ヘルプを出してみる。
$ dnsperf -h DNS Performance Testing Tool Nominum Version 2.1.0.0 Usage: dnsperf [-f family] [-s server_addr] [-p port] [-a local_addr] [-x local_port] [-d datafile] [-c clients] [-T threads] [-n maxruns] [-l timelimit] [-b buffer_size] [-t timeout] [-e] [-D] [-y [alg:]name:secret] [-q num_queries] [-Q max_qps] [-S stats_interval] [-u] [-v] [-h] -f address family of DNS transport, inet or inet6 (default: any) -s the server to query (default: 127.0.0.1) -p the port on which to query the server (default: 53) -a the local address from which to send queries -x the local port from which to send queries (default: 0) -d the input data file (default: stdin) -c the number of clients to act as -T the number of threads to run -n run through input at most N times -l run for at most this many seconds -b socket send/receive buffer size in kilobytes -t the timeout for query completion in seconds (default: 5) -e enable EDNS 0 -D set the DNSSEC OK bit (implies EDNS) -y the TSIG algorithm, name and secret -q the maximum number of queries outstanding (default: 100) -Q limit the number of queries per second -S print qps statistics every N seconds -u send dynamic updates instead of queries -v verbose: report each query to stdout -h print this help
dnsperfより、resperfのが単純負荷テストっぽい感じになりそう。
$ resperf -h DNS Resolution Performance Testing Tool Nominum Version 2.1.0.0 Usage: resperf [-f family] [-s server_addr] [-p port] [-a local_addr] [-x local_port] [-d datafile] [-t timeout] [-b buffer_size] [-e] [-D] [-y [alg:]name:secret] [-i plot_interval] [-m max_qps] [-P plotfile] [-r ramp_time] [-c constant_traffic_time] [-L max_query_loss] [-C clients] [-q num_outstanding] [-h] -f address family of DNS transport, inet or inet6 (default: any) -s the server to query (default: 127.0.0.1) -p the port on which to query the server (default: 53) -a the local address from which to send queries -x the local port from which to send queries (default: 0) -d the input data file (default: stdin) -t the timeout for query completion in seconds (default: 45) -b socket send/receive buffer size in kilobytes -e enable EDNS 0 -D set the DNSSEC OK bit (implies EDNS) -y the TSIG algorithm, name and secret -i the time interval between plot data points, in seconds (default: 0.5) -m the maximum number of queries per second (default: 100000) -P the name of the plot data file (default: resperf.gnuplot) -r the ramp-up time in seconds (default: 60) -c how long to send constant traffic, in seconds (default: 0) -L the maximum acceptable query loss, in percent (default: 100) -C the number of clients to act as -q the maximum number of queries outstanding (default: 65536) -h print this help
query_list_sample.txtを作る。
$ vi query_list_sample.txt a.exsample.com A b.exsample.com A c.exsample.com A
>
負荷の具合は、60秒かけて 100,000QPSというのがデフォルトのようだ。
テスト機のポートは53535を使っているので合わせる。
$ resperf -s localhost -p 53535 -d query_list_sample.txt DNS Resolution Performance Testing Tool Nominum Version 2.1.0.0 [Status] Command line: resperf -s localhost -p 53535 -d query_list_sample.txt [Status] Sending Error: ran out of query data
3行じゃ足りないか... リサイクルはしてくれないのね。
今回は普通のDNSじゃなくて、特定のホスト引くテストでいいんだけどな。
RELEASE_NOTESには、流し込み用のqueryfile-example-100thousandがあるということだけど、ソースアーカイブには含まれてないな。
古いけど、マニュアルがあった。
ftp://ftp.nominum.com/pub/nominum/dnsperf/1.0.1.0/resperf-1.0.1.0-Info-20071218.pdf
今はデータファイルは別配布になっているようだ。
$ wget ftp://ftp.nominum.com/pub/nominum/dnsperf/data/queryfile-example-3million.gz $ gzip -d queryfile-example-3million.gz $ ls -l queryfile-example-3million -rw-r--r--. 1 root root 556 Dec 20 16:45 queryfile-example-3million
なんか小さいな...
$ cat queryfile-example-3million The 3 million record query file has been replaced with a 10 million record query file as 3 million records were not enough for a full resperf run on modern hardware. The latest version of the query file is available at: ftp://ftp.nominum.com/pub/nominum/dnsperf/data/queryfile-example-current.gz If you are using a file that is over a year old, please download an update. Running tests with old query files will skew results.
あっそー。
というか、残しておくなよ。それもgzしてからとかないわー。
$ wget ftp://ftp.nominum.com/pub/nominum/dnsperf/data/queryfile-example-10million-201202.gz $ ls -lh queryfile-example-10million-201202.gz -rw-r--r--. 1 root root 73M Dec 20 16:51 queryfile-example-10million-201202.gz $ gzip -d queryfile-example-10million-201202.gz $ ls -lh queryfile-example-10million-201202 -rw-r--r--. 1 root root 238M Dec 20 16:51 queryfile-example-10million-201202
たいした量じゃなかった。
中身はこんな感じ。
$ wc queryfile-example-10million-201202
10000000 20000000 248609309 queryfile-example-10million-201202
$ head queryfile-example-10million-201202
thumbs2.ebaystatic.com. AAAA
mountaineerpublishing.com. MX
www.mediafire.com. A
s-static.ak.fbcdn.net. A
lachicabionica.com. A
www.freemarket.com. A
sip.hotmail.com. A
www.cangrejas.com. A
google.com. A
cache.defamer.com. A
テストをしてみる。
$ resperf -s localhost -p 53535 -d queryfile-example-10million-201202 DNS Resolution Performance Testing Tool Nominum Version 2.1.0.0 [Status] Command line: resperf -s localhost -p 53535 -d queryfile-example-10million-201202 [Status] Sending [Status] Reached 65536 outstanding queries [Status] Waiting for more responses [Status] Testing complete Statistics: Queries sent: 106819 Queries completed: 41418 Queries lost: 65401 Response codes: NOERROR 41418 (100.00%) Run time (s): 56.334515 Maximum throughput: 5250.000000 qps Lost at that point: 2.81%
60秒かけて 100,000QPSだと、だいぶロストしているな。
毎秒5kクエリ処理できれば性能としては十分だけど。
秒間1000リクエストに絞ってみる。
$ resperf -s localhost -p 53535 -d queryfile-example-10million-201202 -m 1000 DNS Resolution Performance Testing Tool Nominum Version 2.1.0.0 [Status] Command line: resperf -s localhost -p 53535 -d queryfile-example-10million-201202 -m 1000 [Status] Sending [Status] Waiting for more responses [Status] Testing complete Statistics: Queries sent: 30000 Queries completed: 29487 Queries lost: 513 Response codes: NOERROR 29487 (100.00%) Run time (s): 100.000000 Maximum throughput: 1136.000000 qps Lost at that point: 13.15%
毎秒1kだとほとんど落とさない。
800でも傾向は同じ。
Queries sent: 24000 Queries completed: 23579 Queries lost: 421 Response codes: NOERROR 23579 (100.00%) Run time (s): 100.000000 Maximum throughput: 892.000000 qps Lost at that point: 1.11%
5000の場合。
Queries sent: 150000 Queries completed: 125206 Queries lost: 24794 Response codes: NOERROR 125206 (100.00%) Run time (s): 100.000000 Maximum throughput: 4484.000000 qps Lost at that point: 2.61%
一万の場合。
Queries sent: 64618 Queries completed: 58152 Queries lost: 6466 Response codes: NOERROR 58152 (100.00%) Run time (s): 72.422427 Maximum throughput: 4138.000000 qps Lost at that point: 0.48%
同じマシン内でやってることもあるし、UDPバッファのパラメータチューニングがいるかな?
どうせなら、gnuplot入れてグラフにするか。
$ sudo yum install gnuplot (snip) Installed: gnuplot.x86_64 0:4.6.2-3.el7 Dependency Installed: cairo.x86_64 0:1.14.2-1.el7 dejavu-fonts-common.noarch 0:2.33-6.el7 dejavu-sans-fonts.noarch 0:2.33-6.el7 fontconfig.x86_64 0:2.10.95-10.el7 fontpackages-filesystem.noarch 0:1.44-8.el7 gd.x86_64 0:2.0.35-26.el7 gnuplot-common.x86_64 0:4.6.2-3.el7 graphite2.x86_64 0:1.3.6-1.el7_2 harfbuzz.x86_64 0:0.9.36-1.el7 libX11.x86_64 0:1.6.3-3.el7 libX11-common.noarch 0:1.6.3-3.el7 libXau.x86_64 0:1.0.8-2.1.el7 libXdamage.x86_64 0:1.1.4-4.1.el7 libXext.x86_64 0:1.3.3-3.el7 libXfixes.x86_64 0:5.0.1-2.1.el7 libXft.x86_64 0:2.3.2-2.el7 libXpm.x86_64 0:3.5.11-3.el7 libXrender.x86_64 0:0.9.8-2.1.el7 libXxf86vm.x86_64 0:1.1.3-2.1.el7 libjpeg-turbo.x86_64 0:1.2.90-5.el7 libpng.x86_64 2:1.5.13-7.el7_2 libthai.x86_64 0:0.1.14-9.el7 libxcb.x86_64 0:1.11-4.el7 libxshmfence.x86_64 0:1.2-1.el7 mesa-libEGL.x86_64 0:11.2.2-2.20160614.el7 mesa-libGL.x86_64 0:11.2.2-2.20160614.el7 mesa-libgbm.x86_64 0:11.2.2-2.20160614.el7 mesa-libglapi.x86_64 0:11.2.2-2.20160614.el7 pango.x86_64 0:1.36.8-2.el7 pixman.x86_64 0:0.34.0-1.el7
$ resperf-report -s localhost -p 53535 -d queryfile-example-10million-201202 -m 5000 Done, report is in 20161220-1720.html $ cat 20161220-1720.html <html><head></head><body> <h1>Resperf report 20161220-1720</h1> <h2>Resperf output</h2> <pre> DNS Resolution Performance Testing Tool Nominum Version 2.1.0.0 [Status] Command line: resperf -P 20161220-1720.gnuplot -s localhost -p 53535 -d queryfile-example-10million-201202 -m 5000 [Status] Sending [Status] Waiting for more responses [Status] Testing complete Statistics: Queries sent: 150000 Queries completed: 127357 Queries lost: 22643 Response codes: NOERROR 127357 (100.00%) Run time (s): 100.066120 Maximum throughput: 4734.000000 qps Lost at that point: 1.46% </pre> <h2>Plots</h2> <p> <img src="20161220-1720.rate.png" /> <img src="20161220-1720.latency.png" /> </p> </body></html>
ここにBIND9のデータがあるので、マシンスペックわからないけど同じ条件でとってみる。
http://lost-and-found-narihiro.blogspot.jp/2009/08/dns-dnsperf-resperf.html
$ resperf-report -s localhost -p 53535 -d queryfile-example-10million-201202 -m 1000 Done, report is in 20161220-1734.html
BIND9とくらべても、かなりいいレスポンス出ている。
もっとも、すぐに折り返し知らんという返事しているだけで、DBも見ていないだけなんだけど。
一応リクエストのパースまではやっているはずなので、それなりの重さはある。
10000の場合。
遅延はあるけど、とても短い。まだまだ行けそうな感じ。
遅延なしに安定して受けているのは3000弱あたりか。
今後やるかも。
- 同じ環境でBIND上げて比較データ取る
- 同じ環境でAレコードのあるクエリでDBまで降りてくのやってみる
- UDPバッファのパラメータチューニングによる結果の変化調べる