[docker]建立elasticSearch/kibana

本章介绍

建立elasticSearch/kibana [docker]建立index与传送资料查询资料

先备知识:

docker & docker-compose

---

1.启动docker-elasticSearch/kibana

先将网路上找到的docker-compose.yml内容编辑好参考档案
然后在一样的目录下开启指令docker-compose up -d

$docker-compose up -dWARNING: Some services (elasticsearch, kibana) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.Starting elasticsearch-624 ... doneStarting kibana-624        ... done

需要一点时间，可以用Kitematic之类的工具查看有没有成功

确认elasticSearch是否启动

GET localhost:9200 成功会回版号等资讯 "number": "6.2.4"

确认kibana有无成功

开启浏览器http://127.0.0.1:5601/app/kibana#/home?_g=()

---

2.elastic準备资料格式index与新增资料

先设计资料内容，假设今天要收集一个使用者每天的运动纪录
这是一个有array的纪录内容，内容可长可短。

{    "user": "user01",    "timestamp": 1583734521000,    "records": [        {            "record_name": "heart_rate",            "data_number": 80,            "data_txt": "avg"        },        {            "record_name": "Calories",            "data_number": 200        },        {            "record_name": "time_duration",            "data_number": 30,            "record_unit": "min"        }    ]}

接着新增必须栏位的属性index

user是一般text,timestamp是daterecords先建立nested巢状，在建立里面的record_name等栏位。建立index: PUT localhost:9200/{index}

PUT localhost:9200/event{    "mappings": {        "_doc": {            "properties": {                "user": {                    "type": "text",                    "fields": {                        "keyword": {                            "type": "keyword",                            "ignore_above": 256                        }                    }                },                "id": {                    "type": "text",                    "fields": {                        "keyword": {                            "type": "keyword",                            "ignore_above": 256                        }                    }                },                "timestamp": {                    "type": "date",                    "format": "epoch_millis"                },                "records": {                    "type": "nested",                    "properties": {                        "record_name": {                            "type": "text",                            "fields": {                                "keyword": {                                    "type": "keyword",                                    "ignore_above": 256                                }                            }                        },                        "data_number": {                            "type": "long",                            "fields": {                                "keyword": {                                    "type": "keyword",                                    "ignore_above": 256                                }                            }                        },                        "data_txt": {                            "type": "text",                            "fields": {                                "keyword": {                                    "type": "keyword",                                    "ignore_above": 256                                }                            }                        },                        "record_unit": {                            "type": "text",                            "fields": {                                "keyword": {                                    "type": "keyword",                                    "ignore_above": 256                                }                            }                        }                    }                }            }        }    }}

送资料进去

接着把上述资料先送一笔进去http://localhost:9200/{index}/{type}
POST http://localhost:9200/event/_doc

---

kibana建立index

介绍 kibana各个页面使用

建立index

Management页面->create index->填入event->选择可以做时间分割的栏位名称{上述是用timestamp}->按下create index pattern

Discover页面搜寻资料

再送一次资料，这次把 "timestamp": {改成现在时间戳}->线上有很多工具可以做转换
回到Discover页面，query最近15分钟的资料->就可以看到时间轴了

Visualize 建立感兴趣的图表展示

ex: table显示/长条图显示等/或是特定filter资料。然后替图表存档。

Dashboard 页面

这边把刚刚建立的图表拉好显示在这边。

dev tools 页面

透过条件指令搜寻特定资料，如有程式需要可以用搜寻API试着找出自己想搜寻的内容

size/page/sort 分页与排序依据bool query 条件-filter时间/range/match/wildcard等搜寻所有 must 必须匹配，所有 must_not 都必须不匹配minimum_should_match 参数控制需要匹配的 should 语句的数量

範例:

GET /event/_search{  "size": 1000,  "query": {    "bool": {      "filter": {        "range": {          "timestamp": {            "from": 159132465000,            "include_lower": true,            "include_upper": true,            "to": 1591324650099          }        }      },      "must": [        {          "exists": {            "field": "user"          }        },        {          "match": {            "user": {              "operator": "AND",              "query": "user01"            }          }        },        {          "nested": {            "path": "records",            "query": {              "bool": {                "must": [                  {                    "match": {                      "records.record_name": {                        "operator": "AND",                        "query": "heart_rate"                      }                    }                  },                  {                    "wildcard": {                      "records.data_txt": "*a*"                    }                  },                  {                    "range": {                      "records.data_number": {                        "from": 2,                        "include_lower": false,                        "include_upper": true,                        "to": null                      }                    }                  }                ]              }            }          }        }      ],      "minimum_should_match": "1",      "should": [        {          "match": {            "user": {              "operator": "AND",              "query": "user01"            }          }        },        {          "match": {            "user": {              "operator": "AND",              "query": "user02"            }          }        }      ]    }  },  "sort": [    {      "timestamp": {        "order": "desc"      }    }  ]}

---

参考资料

elastic-组合查询 中文

后续未解：elastic该如何建立cluster与管理？？