Finding ElasticSearch

Created by Chris Shayan
Jun 11, 2014
1 min read

It has been quite sometime we are looking into ElasticSearch. Data team of VietnamWorks is doing various activities which is being led by Ninh our Senior Data Architect.

We are moving our whole search system into ElasticSearch hence before making such decision we decided to have a benchmark, ElasticSearch vs MySQL. Ninh setup a load test environment and scenario in our lab at VietnamWorks.

MySQL server: 4 processors ( Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz, cache size 15M) 32G RAM,

ElasticSearch had 1 master node: 4 processors ( Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz, cache size 15M) 8G RAM and 1 data node: 4 processors ( Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz, cache size 15M) 16G RAM

Test Dataset was over 429,700 jobs, on MySQL has stored in 4 tables with 1,822,000 total rows and 1.8G total size and ElasticSearch stored in over 1,285,500 documents (has nested documents), with size 2G.

Our query is quite complicated and it is 3kb file. This query is a search for online jobs with conditions on 14 fields, sort on 4 fields (2 non-stored fields).

Ninh used jMeter 2.11 with following configuration:

  • Number of threads (users): 100
  • Ramp-up period (in seconds): 1
  • Loop count: 2

Run each request separately instead of running both MySQL and ES request in same thread group.

Ninh concluded that ElasticSearch is quite faster than MySQL in our complicated massive query.

 

  • Average response time of ElasticSearch (the best) is faster 33% than this of MySQL.
  • ElasticSearch is more stable than MySQL 48% (Std. dev.).
  • ElasticSearch has no error while MySQL has 1%.
  • ElasticSearch has 33% higher throughput than MySQL .

Thanks to Ninh, we are confidently moving to ElasticSearch. Will keep you all posted ;)