功能特性

實際使用

真題演練1

# 假定有以下索引和數據PUT test03{  "mappings": {    "properties": {      "emotion": {        "type": "integer"      }    }  }}POST test03/_bulk{"index":{"_id":1}}{"emotion":2}{"index":{"_id":2}}{"emotion":5}{"index":{"_id":3}}{"emotion":10}{"index":{"_id":4}}{"emotion":3}# 要求：emotion > 5, 返回emotion_falg = "1",  # 要求：emotion < 5, 返回emotion_falg = "-1",  # 要求：emotion = 5, 返回emotion_falg = "0",

解法1

檢索時指定運行時字段: 該字段本質上是不存在的，所以需要檢索時要加上 fields *

GET test03/_search{  "fields": [    "*"  ],   "runtime_mappings": {    "emotion_falg": {      "type": "keyword",      "script": {        "source": """          if(doc["emotion"].value>5)emit("1");          if(doc["emotion"].value<5)emit("-1");          if(doc["emotion"].value==5)emit("0");          """      }    }  }}

解法2

創建索引時指定運行時字段： 該方式支持通過運行時字段做檢索

# 創建索引并指定運行時字段PUT test03_01{  "mappings": {    "runtime": {      "emotion_falg": {        "type": "keyword",        "script": {          "source": """          if(doc["emotion"].value>5)emit("1");          if(doc["emotion"].value<5)emit("-1");          if(doc["emotion"].value==5)emit("0");          """        }      }    },    "properties": {      "emotion": {        "type": "integer"      }    }  }}# 導入測試數據POST test03_01/_bulk{"index":{"_id":1}}{"emotion":2}{"index":{"_id":2}}{"emotion":5}{"index":{"_id":3}}{"emotion":10}{"index":{"_id":4}}{"emotion":3}# 查詢測試GET test03_01/_search{  "fields": [    "*"  ]}

真題演練2

# 有以下索引和數據PUT test04{  "mappings": {    "properties": {      "A":{        "type": "long"      },      "B":{        "type": "long"      }    }  }}PUT task04/_bulk{"index":{"_id":1}}{"A":100,"B":2}{"index":{"_id":2}}{"A":120,"B":2}{"index":{"_id":3}}{"A":120,"B":25}{"index":{"_id":4}}{"A":21,"B":25}# 需求：在task04索引里，創建一個runtime字段，其值是A-B，名稱為A_B； 創建一個range聚合，分為三級：小于0，0-100，100以上；返回文檔數// 使用知識點：// 1.檢索時指定運行時字段: https://www.elastic.co/guide/en/elasticsearch/reference/8.1/runtime-search-request.html// 2.范圍聚合 https://www.elastic.co/guide/en/elasticsearch/reference/8.1/search-aggregations-bucket-range-aggregation.html

解法

# 結果測試GET task04/_search{  "fields": [    "*"  ],   "size": 0,   "runtime_mappings": {    "A_B": {      "type": "long",      "script": {        "source": """          emit(doc["A"].value - doc["B"].value);          """      }    }  },  "aggs": {    "price_ranges_A_B": {      "range": {        "field": "A_B",        "ranges": [          { "to": 0 },          { "from": 0, "to": 100 },          { "from": 100 }        ]      }    }  }}

八.Search-highlighted

highlighted語法初識

官網文檔地址：??https://www.elastic.co/guide/en/elasticsearch/reference/8.1/highlighting.html??

九.Search-Order

Order語法初識

官網文檔地址： ??https://www.elastic.co/guide/en/elasticsearch/reference/8.1/sort-search-results.html??

// 注意：text類型默認是不能排或聚合的，如果非要排序或聚合，需要開啟fielddataGET /kibana_sample_data_ecommerce/_search{  "query": {    "match": {      "customer_last_name": "wood"    }  },  "highlight": {    "number_of_fragments": 3,    "fragment_size": 150,    "fields": {      "customer_last_name": {        "pre_tags": [          ""        ],        "post_tags": [          ""        ]      }    }  },  "sort": [    {      "currency": {        "order": "desc"      },      "_score": {        "order": "asc"      }    }  ]}

十.Search-Page

page語法初識

官網文檔地址：??https://www.elastic.co/guide/en/elasticsearch/reference/8.1/paginate-search-results.html??

# 注意 from的起始值是 0 不是 1GET kibana_sample_data_ecommerce/_search{  "from": 5,  "size": 20,  "query": {    "match": {      "customer_last_name": "wood"    }  }}

真題演練1

# 題目In the spoken lines of the play, highlight the word Hamlet (int the text_entry field) startint the highlihnt with "#aaa#" and ending it with "#bbb#"return all of speech_number field lines in reverse order; "20" speech lines per page,starting from line "40"# highlight 處理 text_entry 字段 ； 關鍵詞 Hamlet 高亮# page分頁：from：40；size:20# speech_number：倒序POST test09/_search{  "from": 40,  "size": 20,  "query": {    "bool": {      "must": [        {          "match": {            "text_entry": "Hamlet"          }        }      ]    }  },  "highlight": {    "fields": {      "text_entry": {        "pre_tags": [          "#aaa#"        ],        "post_tags": [          "#bbb#"        ]      }    }  },  "sort": [    {      "speech_number.keyword": {        "order": "desc"      }    }  ]}

十一.Search-AsyncSearch

官網文檔地址：??https://www.elastic.co/guide/en/elasticsearch/reference/8.1/async-search.html??

發行版本

7.7.0

適用場景

允許用戶在異步搜索結果時可以檢索，從而消除了僅在查詢完成后才等待最終響應的情況

常用命令

異步查詢結果說明

十二.Aliases索引別名

官網文檔地址：??https://www.elastic.co/guide/en/elasticsearch/reference/8.1/aliases.html??

Aliases的作用

在ES中，索引別名（index aliases）就像一個快捷方式或??軟連接??，可以指向一個或多個索引。別名帶給我們極大的靈活性，我們可以使用索引別名實現以下功能：

假設沒有別名，如何處理多索引的檢索

方式1：POST index_01,index_02.index_03/_search 方式2：POST index*/search

創建別名的三種方式

# 指定test05的別名為 test05_aliasesPUT test05{  "mappings": {    "properties": {      "name":{        "type": "keyword"      }    }  },  "aliases": {    "test05_aliases": {}  }}

PUT _index_template/template_1{  "index_patterns": ["te*", "bar*"],  "template": {    "settings": {      "number_of_shards": 1    },    "mappings": {      "_source": {        "enabled": true      },      "properties": {        "host_name": {          "type": "keyword"        },        "created_at": {          "type": "date",          "format": "EEE MMM dd HH:mm:ss Z yyyy"        }      }    },    "aliases": {      "mydata": { }    }  },  "priority": 500,  "composed_of": ["component_template1", "runtime_component_template"],   "version": 3,  "_meta": {    "description": "my custom"  }}

POST _aliases{  "actions": [    {      "add": {        "index": "logs-nginx.access-prod",        "alias": "logs"      }    }  ]}

刪除別名

POST _aliases{  "actions": [    {      "remove": {        "index": "logs-nginx.access-prod",        "alias": "logs"      }    }  ]}

真題演練1

# Define an index alias for "accounts-row" called "accounts-male": Apply a filter to only show the male account owners# 為"accounts-row"定義一個索引別名，稱為"accounts-male":應用一個過濾器，只顯示男性賬戶所有者POST _aliases{  "actions": [    {      "add": {        "index": "accounts-row",        "alias": "accounts-male",        "filter": {          "bool": {            "filter": [              {                "term": {                  "gender.keyword": "male"                }              }            ]          }        }      }    }  ]}

十三.Search-template

官網文檔地址:??https://www.elastic.co/guide/en/elasticsearch/reference/8.1/search-template.html??

功能特點

模板接受在運行時指定參數。搜索模板存儲在服務器端，可以在不更改客戶端代碼的情況下進行修改。

初識search-template

# 創建檢索模板PUT _scripts/my-search-template{  "script": {    "lang": "mustache",    "source": {      "query": {        "match": {          "{{query_key}}": "{{query_value}}"        }      },      "from": "{{from}}",      "size": "{{size}}"    }  }}# 使用檢索模板查詢GET my-index/_search/template{  "id": "my-search-template",  "params": {    "query_key": "your filed",    "query_value": "your filed value",    "from": 0,    "size": 10  }}

索引模板的操作

創建索引模板

PUT _scripts/my-search-template{  "script": {    "lang": "mustache",    "source": {      "query": {        "match": {          "message": "{{query_string}}"        }      },      "from": "{{from}}",      "size": "{{size}}"    },    "params": {      "query_string": "My query string"    }  }}

驗證索引模板

POST _render/template{  "id": "my-search-template",  "params": {    "query_string": "hello world",    "from": 20,    "size": 10  }}

執行檢索模板

GET my-index/_search/template{  "id": "my-search-template",  "params": {    "query_string": "hello world",    "from": 0,    "size": 10  }}

獲取全部檢索模板

GET _cluster/state/metadata?pretty&filter_path=metadata.stored_scripts

刪除檢索模板

DELETE _scripts/my-search-templateath=metadata.stored_scripts

十四.Search-dsl 簡單檢索

官網文檔地址：??https://www.elastic.co/guide/en/elasticsearch/reference/8.1/query-dsl.html??

檢索選型

檢索分類

自定義評分

如何自定義評分

1.index Boost索引層面修改相關性

// 一批數據里，有不同的標簽，數據結構一致，不同的標簽存儲到不同的索引（A、B、C），最后要嚴格按照標簽來分類展示的話，用什么查詢比較好?// 要求：先展示A類，然后B類，然后C類# 測試數據如下put /index_a_123/_doc/1{  "title":"this is index_a..."}put /index_b_123/_doc/1{  "title":"this is index_b..."}put /index_c_123/_doc/1{  "title":"this is index_c..."}# 普通不指定的查詢方式，該查詢方式下，返回的三條結果數據評分是相同的POST index_*_123/_search{  "query": {    "bool": {      "must": [        {          "match": {            "title": "this"          }        }      ]    }  }}官網文檔地址：https://www.elastic.co/guide/en/elasticsearch/reference/8.1/search-search.htmlindices_boost# 也就是索引層面提升權重POST index_*_123/_search{  "indices_boost": [    {      "index_a_123": 10    },    {      "index_b_123": 5    },    {      "index_c_123": 1    }  ],   "query": {    "bool": {      "must": [        {          "match": {            "title": "this"          }        }      ]    }  }}

2.boosting 修改文檔相關性

某索引index_a有多個字段， 要求實現如下的查詢：1）針對字段title，滿足"ssas"或者"sasa’。2）針對字段tags（數組字段），如果tags字段包含"pingpang",則提升評分。要求：寫出實現的DSL？# 測試數據如下put index_a/_bulk{"index":{"_id":1}}{"title":"ssas","tags":"basketball"}{"index":{"_id":2}}{"title":"sasa","tags":"pingpang; football"}# 解法1POST index_a/_search{  "query": {    "bool": {      "must": [        {          "bool": {            "should": [              {                "match": {                  "title": "ssas"                }              },              {                "match": {                  "title": "sasa"                }              }            ]          }        }      ],      "should": [        {          "match": {            "tags": {              "query": "pingpang",              "boost": 1            }                      }        }      ]    }  }}# 解法2// https://www.elastic.co/guide/en/elasticsearch/reference/8.1/query-dsl-function-score-query.htmlPOST index_a/_search{  "query": {    "bool": {      "should": [        {          "function_score": {            "query": {              "match": {                "tags": {                  "query": "pingpang"                }              }            },            "boost": 1          }        }      ],      "must": [        {          "bool": {            "should": [              {                "match": {                  "title": "ssas"                }              },              {                "match": {                  "title": "sasa"                }              }            ]          }        }      ]    }  }}

3.negative_boost降低相關性

對于某些結果不滿意，但又不想通過 must_not 排除掉，可以考慮可以考慮boosting query的negative_boost。即：降低評分negative_boost(Required, float) Floating point number between 0 and 1.0 used to decrease the relevance scores of documents matching the negative query.官網文檔地址：https://www.elastic.co/guide/en/elasticsearch/reference/8.1/query-dsl-boosting-query.htmlPOST index_a/_search{  "query": {    "boosting": {      "positive": {        "term": {          "tags": "football"        }      },      "negative": {        "term": {          "tags": "pingpang"        }      },      "negative_boost": 0.5    }  }}

4.function_score 自定義評分

如何同時根據 銷量和瀏覽人數進行相關度提升？問題描述：針對商品，例如有想要有一個提升相關度的計算，同時針對銷量和瀏覽人數？例如oldScore*(銷量+瀏覽人數)**************************  商品        銷量        瀏覽人數  A         10           10      B         20           20C         30           30************************** # 示例數據如下    put goods_index/_bulk{"index":{"_id":1}}{"name":"A","sales_count":10,"view_count":10}{"index":{"_id":2}}{"name":"B","sales_count":20,"view_count":20}{"index":{"_id":3}}{"name":"C","sales_count":30,"view_count":30}官網文檔地址：https://www.elastic.co/guide/en/elasticsearch/reference/8.1/query-dsl-function-score-query.html知識點：script_scorePOST goods_index/_search{  "query": {    "function_score": {      "query": {        "match_all": {}      },      "script_score": {        "script": {          "source": "_score * (doc["sales_count"].value+doc["view_count"].value)"        }      }    }  }}

十五.Search-del Bool復雜檢索

官網文檔地址：??https://www.elastic.co/guide/en/elasticsearch/reference/8.1/query-dsl-bool-query.html??

基本語法

真題演練

寫一個查詢，要求某個關鍵字再文檔的四個字段中至少包含兩個以上功能點：bool 查詢，should / minimum_should_match    1.檢索的bool查詢    2.細節點 minimum_should_match注意：minimum_should_match 當有其他子句的時候，默認值為0，當沒有其他子句的時候默認值為1POST test_index/_search{  "query": {    "bool": {      "should": [        {          "match": {            "filed1": "kr"          }        },        {          "match": {            "filed2": "kr"          }        },        {          "match": {            "filed3": "kr"          }        },        {          "match": {            "filed4": "kr"          }        }      ],      "minimum_should_match": 2    }  }}

十六.Search-Aggregations

官網文檔地址：??https://www.elastic.co/guide/en/elasticsearch/reference/8.1/search-aggregations.html??

聚合分類

分桶聚合（bucket）

terms

官網文檔地址：https://www.elastic.co/guide/en/elasticsearch/reference/8.1/search-aggregations-bucket-terms-aggregation.html# 按照作者統計文檔數POST bilili_elasticsearch/_search{  "size": 0,  "aggs": {    "agg_user": {      "terms": {        "field": "user",        "size": 1      }    }  }}

date_histogram

官網文檔地址：https://www.elastic.co/guide/en/elasticsearch/reference/8.1/search-aggregations-bucket-datehistogram-aggregation.html# 按照up_time 按月進行統計POST bilili_elasticsearch/_search{  "size": 0,  "aggs": {    "agg_up_time": {      "date_histogram": {        "field": "up_time",        "calendar_interval": "month"      }    }  }}

指標聚合 （metrics）

Max

官網文檔地址：https://www.elastic.co/guide/en/elasticsearch/reference/8.1/search-aggregations-metrics-max-aggregation.html# 獲取up_time最大的POST bilili_elasticsearch/_search{  "size": 0,  "aggs": {    "agg_max_up_time": {      "max": {        "field": "up_time"      }    }  }}

Top_hits

官網文檔地址：https://www.elastic.co/guide/en/elasticsearch/reference/8.1/search-aggregations-metrics-top-hits-aggregation.html# 根據user聚合只取一個聚合結果，并且獲取命中數據的詳情前3條，并按照指定字段排序POST bilili_elasticsearch/_search{  "size": 0,  "aggs": {    "terms_agg_user": {      "terms": {        "field": "user",        "size": 1      },      "aggs": {        "top_user_hits": {          "top_hits": {            "_source": {              "includes": [                "video_time",                "title",                "see",                "user",                "up_time"              ]            },             "sort": [              {                "see":{                  "order": "desc"                }              }            ],             "size": 3          }        }      }    }  }}// 返回結果如下{  "took" : 91,  "timed_out" : false,  "_shards" : {    "total" : 1,    "successful" : 1,    "skipped" : 0,    "failed" : 0  },  "hits" : {    "total" : {      "value" : 1000,      "relation" : "eq"    },    "max_score" : null,    "hits" : [ ]  },  "aggregations" : {    "terms_agg_user" : {      "doc_count_error_upper_bound" : 0,      "sum_other_doc_count" : 975,      "buckets" : [        {          "key" : "Elastic搜索",          "doc_count" : 25,          "top_user_hits" : {            "hits" : {              "total" : {                "value" : 25,                "relation" : "eq"              },              "max_score" : null,              "hits" : [                {                  "_index" : "bilili_elasticsearch",                  "_id" : "5ccCVoQBUyqsIDX6wIcm",                  "_score" : null,                  "_source" : {                    "video_time" : "03:45",                    "see" : "92",                    "up_time" : "2021-03-19",                    "title" : "Elastic 社區大會2021: 用加 Gatling 進行Elasticsearch的負載測試，寓教于樂。",                    "user" : "Elastic搜索"                  },                  "sort" : [                    "92"                  ]                },                {                  "_index" : "bilili_elasticsearch",                  "_id" : "8scCVoQBUyqsIDX6wIgn",                  "_score" : null,                  "_source" : {                    "video_time" : "10:18",                    "see" : "79",                    "up_time" : "2020-10-20",                    "title" : "為Elasticsearch啟動htpps訪問",                    "user" : "Elastic搜索"                  },                  "sort" : [                    "79"                  ]                },                {                  "_index" : "bilili_elasticsearch",                  "_id" : "7scCVoQBUyqsIDX6wIcm",                  "_score" : null,                  "_source" : {                    "video_time" : "04:41",                    "see" : "71",                    "up_time" : "2021-03-19",                    "title" : "Elastic 社區大會2021: Elasticsearch作為一個地理空間的數據庫",                    "user" : "Elastic搜索"                  },                  "sort" : [                    "71"                  ]                }              ]            }          }        }      ]    }  }}

子聚合 （Pipeline）

Pipeline：基于聚合的聚合 官網文檔地址：??https://www.elastic.co/guide/en/elasticsearch/reference/8.1/search-aggregations-pipeline.html??

bucket_selector

官網文檔地址：??https://www.elastic.co/guide/en/elasticsearch/reference/8.1/search-aggregations-pipeline-bucket-selector-aggregation.html??

# 根據order_date按月分組，并且求銷售總額大于1000POST kibana_sample_data_ecommerce/_search{  "size": 0,  "aggs": {    "date_his_aggs": {      "date_histogram": {        "field": "order_date",        "calendar_interval": "month"      },      "aggs": {        "sum_aggs": {          "sum": {            "field": "total_unique_products"          }        },        "sales_bucket_filter": {          "bucket_selector": {            "buckets_path": {              "totalSales": "sum_aggs"            },            "script": "params.totalSales > 1000"          }        }      }    }  }}

真題演練

earthquakes索引中包含了過去30個月的地震信息，請通過一句查詢，獲取以下信息l 過去30個月，每個月的平均 magl 過去30個月里，平均mag最高的一個月及其平均magl 搜索不能返回任何文檔    max_bucket 官網地址：https://www.elastic.co/guide/en/elasticsearch/reference/8.1/search-aggregations-pipeline-max-bucket-aggregation.htmlPOST earthquakes/_search{  "size": 0,   "query": {    "range": {      "time": {        "gte": "now-30M/d",        "lte": "now"      }    }  },  "aggs": {    "agg_time_his": {      "date_histogram": {        "field": "time",        "calendar_interval": "month"      },      "aggs": {        "avg_aggs": {          "avg": {            "field": "mag"          }        }      }    },    "max_mag_sales": {      "max_bucket": {        "buckets_path": "agg_time_his>avg_aggs"       }    }  }}