2014年5月6日 星期二

[elasticsearch]use Aggregations to do pivot table (樞紐分析表)


一般可能會比較 solr 有 pivot table 的這個用法,
其實在elastic search 裡面可以使用 facet search 做替代。

在 1.x以後 加入新的 Aggregations ,我們可以使用他來產生 pivot table。

ex:
1.先index 一些 records
curl -XPOST 'http://localhost:9200/twitter/tweet/' -d '{
                    "user": "kimchy",
                    "post_date": "2009-11-15T14:12:12",
                    "message": "trying out Elasticsearch",
                    "ip": "123.0.0.2",
                    "groups": "B"
 }'
{
                    "user": "kimchy",
                    "post_date": "2009-11-15T14:12:12",
                    "message": "trying out Elasticsearch",
                    "ip": "123.0.0.2",
                    "groups": "A"
   }
 {
                    "user": "pc",
                    "post_date": "2009-11-15T14:12:12",
                    "message": "trying out Elasticsearch",
                    "ip": "123.0.0.5",
                    "groups": "C"
                }
{
                    "user": "kimchy",
                    "post_date": "2009-11-15T14:12:12",
                    "message": "trying out Elasticsearch",
                    "ip": "123.0.0.2",
                    "groups": "C"
                }
{
                  "user": "kimchy",
                    "post_date": "2009-11-15T14:12:12",
                    "message": "trying out Elasticsearch",
                    "ip": "123.0.0.1",
                    "groups": "A"
                }
{
                    "user": "pc",
                    "post_date": "2009-11-15T14:12:12",
                    "message": "trying out Elasticsearch",
                    "ip": "123.0.0.5",
                    "groups": "A"
                }
          {
                    "user": "pc",
                    "post_date": "2009-11-15T14:12:12",
                    "message": "trying out Elasticsearch",
                    "ip": "123.0.0.2",
                    "groups": "C"
                }
            {
                    "user": "kimchy2",
                    "post_date": "2009-11-15T14:12:12",
                    "message": "trying out Elasticsearch",
                    "ip": "123.0.0.2",
                    "groups": "A"
                }

curl -XPOST http://localhost:9200/twitter/_search?pretty=true -D '
{
    "aggregations": {
        "maincategory": {
            "aggregations": {
                "main2category": {
                    "aggregations": {
                        "main3category": {
                            "terms": {
                                "field": "user"
                            }
                        }
                    },
                    "terms": {
                        "field": "ip"
                    }
                }
            },
            "terms": {
                "field": "groups"
            }
        }
    }
}
'

得到的結果
{
    "took": 66,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 8,
        "max_score": 1,
        "hits": [
            {
                "_index": "twitter",
                "_type": "tweet",
                "_id": "XgQFw9ncTf2LhIOdb3Jw8g",
                "_score": 1,
                "_source": {
                    "user": "kimchy",
                    "post_date": "2009-11-15T14:12:12",
                    "message": "trying out Elasticsearch",
                    "ip": "123.0.0.2",
                    "groups": "B"
                }
            },
            {
                "_index": "twitter",
                "_type": "tweet",
                "_id": "BEN-DAVLSpacVvViGl1RxQ",
                "_score": 1,
                "_source": {
                    "user": "kimchy",
                    "post_date": "2009-11-15T14:12:12",
                    "message": "trying out Elasticsearch",
                    "ip": "123.0.0.2",
                    "groups": "A"
                }
            },
            {
                "_index": "twitter",
                "_type": "tweet",
                "_id": "GByKrOpJRnCeWtVpvKo1bA",
                "_score": 1,
                "_source": {
                    "user": "pc",
                    "post_date": "2009-11-15T14:12:12",
                    "message": "trying out Elasticsearch",
                    "ip": "123.0.0.5",
                    "groups": "A"
                }
            },
            {
                "_index": "twitter",
                "_type": "tweet",
                "_id": "5ExMEa4TQMKyLyBR-61CUg",
                "_score": 1,
                "_source": {
                    "user": "kimchy",
                    "post_date": "2009-11-15T14:12:12",
                    "message": "trying out Elasticsearch",
                    "ip": "123.0.0.1",
                    "groups": "A"
                }
            },
            {
                "_index": "twitter",
                "_type": "tweet",
                "_id": "gMHOz5DqRaiL15dIVh_nzA",
                "_score": 1,
                "_source": {
                    "user": "pc",
                    "post_date": "2009-11-15T14:12:12",
                    "message": "trying out Elasticsearch",
                    "ip": "123.0.0.2",
                    "groups": "C"
                }
            },
            {
                "_index": "twitter",
                "_type": "tweet",
                "_id": "XJHgowtKQ26zvUj2Md4b4g",
                "_score": 1,
                "_source": {
                    "user": "kimchy2",
                    "post_date": "2009-11-15T14:12:12",
                    "message": "trying out Elasticsearch",
                    "ip": "123.0.0.2",
                    "groups": "A"
                }
            },
            {
                "_index": "twitter",
                "_type": "tweet",
                "_id": "2JjnAJxrRJ6JPCNLhlgYfQ",
                "_score": 1,
                "_source": {
                    "user": "kimchy",
                    "post_date": "2009-11-15T14:12:12",
                    "message": "trying out Elasticsearch",
                    "ip": "123.0.0.2",
                    "groups": "C"
                }
            },
            {
                "_index": "twitter",
                "_type": "tweet",
                "_id": "_xPuE9bPTr2p2IKIpVDKSw",
                "_score": 1,
                "_source": {
                    "user": "pc",
                    "post_date": "2009-11-15T14:12:12",
                    "message": "trying out Elasticsearch",
                    "ip": "123.0.0.5",
                    "groups": "C"
                }
            }
        ]
    },
    "aggregations": {
        "maincategory": {
            "buckets": [
                {
                    "key": "a",
                    "doc_count": 4,
                    "main2category": {
                        "buckets": [
                            {
                                "key": "123.0.0.2",
                                "doc_count": 2,
                                "main3category": {
                                    "buckets": [
                                        {
                                            "key": "kimchy",
                                            "doc_count": 1
                                        },
                                        {
                                            "key": "kimchy2",
                                            "doc_count": 1
                                        }
                                    ]
                                }
                            },
                            {
                                "key": "123.0.0.1",
                                "doc_count": 1,
                                "main3category": {
                                    "buckets": [
                                        {
                                            "key": "kimchy",
                                            "doc_count": 1
                                        }
                                    ]
                                }
                            },
                            {
                                "key": "123.0.0.5",
                                "doc_count": 1,
                                "main3category": {
                                    "buckets": [
                                        {
                                            "key": "pc",
                                            "doc_count": 1
                                        }
                                    ]
                                }
                            }
                        ]
                    }
                },
                {
                    "key": "c",
                    "doc_count": 3,
                    "main2category": {
                        "buckets": [
                            {
                                "key": "123.0.0.2",
                                "doc_count": 2,
                                "main3category": {
                                    "buckets": [
                                        {
                                            "key": "kimchy",
                                            "doc_count": 1
                                        },
                                        {
                                            "key": "pc",
                                            "doc_count": 1
                                        }
                                    ]
                                }
                            },
                            {
                                "key": "123.0.0.5",
                                "doc_count": 1,
                                "main3category": {
                                    "buckets": [
                                        {
                                            "key": "pc",
                                            "doc_count": 1
                                        }
                                    ]
                                }
                            }
                        ]
                    }
                },
                {
                    "key": "b",
                    "doc_count": 1,
                    "main2category": {
                        "buckets": [
                            {
                                "key": "123.0.0.2",
                                "doc_count": 1,
                                "main3category": {
                                    "buckets": [
                                        {
                                            "key": "kimchy",
                                            "doc_count": 1
                                        }
                                    ]
                                }
                            }
                        ]
                    }
                }
            ]
        }
    }
}


DevMynd Blog: GeoHash Grid Aggregation with ElasticSearch
http://www.devmynd.com/blog/2014-2-geohash-grid-aggregation-with-elasticsearch
Elasticsearch's New Aggregations
https://www.found.no/foundation/elasticsearch-aggregations/
Elasticsearch Aggregations Overview | Chris Simpson - Software Developer
http://chrissimpson.co.uk/elasticsearch-aggregations-overview.html
Aggregations
http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/search-aggregations.html

沒有留言:

張貼留言