投稿
  1. 云图网首页
  2. 技术专区
  3. 编程笔记

ES命令行

• 编程笔记 • 阅读 109


REST APIs

Index management

Create index API

创建新索引

PUT /twitter

curl -X PUT "localhost:9200/twitter?pretty"

索引设置：创建的每个索引都可以有与之关联的特定设置。

PUT /twitter
{
    "settings" : {
        "index" : {
            "number_of_shards" : 3, 
            "number_of_replicas" : 2 
        }
    }
}

curl -X PUT "localhost:9200/twitter?pretty" -H 'Content-Type: application/json' -d'
{
    "settings" : {
        "index" : {
            "number_of_shards" : 3, 
            "number_of_replicas" : 2 
        }
    }
}
'

索引设置-简化

PUT /twitter
{
    "settings" : {
        "number_of_shards" : 3,
        "number_of_replicas" : 2
    }
}

curl -X PUT "localhost:9200/twitter?pretty" -H 'Content-Type: application/json' -d'
{
    "settings" : {
        "number_of_shards" : 3,
        "number_of_replicas" : 2
    }
}
'

mappings：映射

PUT /test
{
    "settings" : {
        "number_of_shards" : 1
    },
    "mappings" : {
        "properties" : {
            "field1" : { "type" : "text" }
        }
    }
}

curl -X PUT "localhost:9200/test?pretty" -H 'Content-Type: application/json' -d'
{
    "settings" : {
        "number_of_shards" : 1
    },
    "mappings" : {
        "properties" : {
            "field1" : { "type" : "text" }
        }
    }
}
'

Aliases：别名

PUT /test
{
    "aliases" : {
        "alias_1" : {},
        "alias_2" : {
            "filter" : {
                "term" : {"user" : "kimchy" }
            },
            "routing" : "kimchy"
        }
    }
}

curl -X PUT "localhost:9200/test?pretty" -H 'Content-Type: application/json' -d'
{
    "aliases" : {
        "alias_1" : {},
        "alias_2" : {
            "filter" : {
                "term" : {"user" : "kimchy" }
            },
            "routing" : "kimchy"
        }
    }
}
'

Wait For active shards
默认情况下，索引创建只会在每个分片的主副本启动或请求超时时向客户端返回响应。

{
    "acknowledged": true,
    "shards_acknowledged": true,
    "index": "test"
}
  • acknowledged：指示是否在集群中成功创建了索引
  • shards_acknowledged：指示是否在超时之前为索引中的每个分片启动了所需数量的分片副本。
  • 请注意，acknowledged 或 shards_acknowledged 仍然可能为 false，但索引创建成功。这些值仅指示操作是否在超时之前完成。
  • 如果acknowledge为false，那么我们在用新创建的索引更新集群状态之前就超时了，但它可能很快就会被创建。
  • 如果shards_acknowledged为false，那么我们在启动所需数量的分片之前超时（默认情况下只是主分片），即使集群状态已成功更新以反映新创建的索引（即acknowledged=true）。

改变只等待主分片启动的默认值：改变这个设置也会影响后续所有写操作的wait_for_active_shards值

  • 方法一：通过索引设置index.write.wait_for_active_shards
PUT /test
{
    "settings": {
        "index.write.wait_for_active_shards": "2"
    }
}

curl -X PUT "localhost:9200/test?pretty" -H 'Content-Type: application/json' -d'
{
    "settings": {
        "index.write.wait_for_active_shards": "2"
    }
}
'
  • 方法二：通过请求参数wait_for_active_shards
PUT /test?wait_for_active_shards=2

curl -X PUT "localhost:9200/test?wait_for_active_shards=2&pretty"

Delete index API

删除指定索引

DELETE /twitter

curl -X DELETE "localhost:9200/twitter?pretty"

删除多个索引，中间用逗号隔开

curl -X DELETE http://localhost:9200/索引名称1,索引名称2

模糊匹配删除

curl -X DELETE -u elastic:changeme http://localhost:9200/索引名前缀*

使用通配符，删除所有索引

curl -X DELETE http://localhost:9200/_all
curl -X DELETE http://localhost:9200/*

定时删除ES索引

30 2 * * * /usr/bin/curl -X DELETE http://localhost:9200/*-$(date -d '-3days' +'%Y.%m.%d') >/dev/null 2>&1

定时删除脚本

#!/bin/bash
time=$(date -d '-3days' +'%Y.%m.%d')
curl -X DELETE -u elastic:changeme http://localhost:9200/*-${time}

Get index API

获取一个或多个索引信息

GET /twitter

curl -X GET "localhost:9200/twitter?pretty"

Index exists API

检查索引是否存在。

HEAD /twitter

curl -I "localhost:9200/twitter?pretty"
  • 200：表示存在所有指定的索引或索引别名。
  • 404：表示一个或多个指定的索引或索引别名不存在。

Open Index API

POST /twitter/_open

curl -X POST "localhost:9200/twitter/_open?pretty"

Close Index API

POST /twitter/_close

curl -X POST "localhost:9200/twitter/_close?pretty"

Shrink index API

缩小索引：将现有索引收缩为具有较少主分片的新索引，例：my_source_index缩小为名为my_target_index的新索引。

POST /my_source_index/_shrink/my_target_index
{
  "settings": {
    "index.routing.allocation.require._name": null, //清除从源索引复制的分配要求。
    "index.blocks.write": null //清除从源索引复制的索引写入块。
  }
}

curl -X POST "localhost:9200/my_source_index/_shrink/my_target_index?pretty" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "index.routing.allocation.require._name": null, 
    "index.blocks.write": null 
  }
}
'

前提条件：可通过请求实现

  • 索引必须是只读的。
  • 索引中每个分片的副本必须驻留在同一个节点上。
  • 集群健康状态必须为绿色。
PUT /my_source_index/_settings
{
  "settings": {
    "index.routing.allocation.require._name": "shrink_node_name", 
    "index.blocks.write": true 
  }
}

curl -X PUT "localhost:9200/my_source_index/_settings?pretty" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "index.routing.allocation.require._name": "shrink_node_name", 
    "index.blocks.write": true 
  }
}
'

settings+aliases：类似create index API，接受目标索引的设置和别名参数

POST /my_source_index/_shrink/my_target_index
{
  "settings": {
    "index.number_of_replicas": 1,
    "index.number_of_shards": 1, 
    "index.codec": "best_compression" 
  },
  "aliases": {
    "my_search_indices": {}
  }
}

curl -X POST "localhost:9200/my_source_index/_shrink/my_target_index?pretty" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "index.number_of_replicas": 1,
    "index.number_of_shards": 1, 
    "index.codec": "best_compression" 
  },
  "aliases": {
    "my_search_indices": {}
  }
}
'

Alias management

Add index alias API

创建或更新索引别名。

  • 索引别名是用于引用一个或多个现有索引的辅助名称。
  • 大多数 Elasticsearch API 接受索引别名来代替索引名称。
PUT /twitter/_alias/alias1

curl -X PUT "localhost:9200/twitter/_alias/alias1?pretty"

添加基于时间的别名

  • 以下请求为 logs_20302801 索引创建别名 2030。
PUT /logs_20302801/_alias/2030

curl -X PUT "localhost:9200/logs_20302801/_alias/2030?pretty"

添加基于用户的别名

  • 首先，使用 user_id 字段的映射创建一个索引 users
PUT /users
{
    "mappings" : {
        "properties" : {
            "user_id" : {"type" : "integer"}
        }
    }
}

curl -X PUT "localhost:9200/users?pretty" -H 'Content-Type: application/json' -d'
{
    "mappings" : {
        "properties" : {
            "user_id" : {"type" : "integer"}
        }
    }
}
'
  • 然后为特定用户 user_12 添加索引别名：
PUT /users/_alias/user_12
{
    "routing" : "12",
    "filter" : {
        "term" : {
            "user_id" : 12
        }
    }
}

curl -X PUT "localhost:9200/users/_alias/user_12?pretty" -H 'Content-Type: application/json' -d'
{
    "routing" : "12",
    "filter" : {
        "term" : {
            "user_id" : 12
        }
    }
}
'

创建索引时添加别名
可以使用 create index API 在索引创建过程中添加索引别名。

PUT /logs_20302801
{
    "mappings" : {
        "properties" : {
            "year" : {"type" : "integer"}
        }
    },
    "aliases" : {
        "current_day" : {},
        "2030" : {
            "filter" : {
                "term" : {"year" : 2030 }
            }
        }
    }
}

curl -X PUT "localhost:9200/logs_20302801?pretty" -H 'Content-Type: application/json' -d'
{
    "mappings" : {
        "properties" : {
            "year" : {"type" : "integer"}
        }
    },
    "aliases" : {
        "current_day" : {},
        "2030" : {
            "filter" : {
                "term" : {"year" : 2030 }
            }
        }
    }
}
'

Index alias exists API

检查索引别名是否存在。

  • 索引别名是用于引用一个或多个现有索引的辅助名称。
  • 大多数 Elasticsearch API 接受索引别名来代替索引名称。
HEAD /_alias/alias1

curl -I "localhost:9200/_alias/alias1?pretty"

Get index alias API

返回有关一个或多个索引别名的信息。

  • 索引别名是用于引用一个或多个现有索引的辅助名称。
  • 大多数 Elasticsearch API 接受索引别名来代替索引名称。
GET /twitter/_alias/alias1

curl -X GET "localhost:9200/twitter/_alias/alias1?pretty"

Monitoring

Index stats

单索引状态

GET /twitter/_stats

curl -X GET "localhost:9200/twitter/_stats?pretty"

多索引状态

GET /index1,index2/_stats

curl -X GET "localhost:9200/index1,index2/_stats?pretty"

全部索引状态

GET /_stats

curl -X GET "localhost:9200/_stats?pretty"

返回所有索引的合并和刷新统计信息。

GET /_stats/merge,refresh

curl -X GET "localhost:9200/_stats/merge,refresh?pretty"

返回 group1 和 group2 搜索组的搜索统计信息。

GET /_stats/search?groups=group1,group2

curl -X GET "localhost:9200/_stats/search?groups=group1,group2&pretty"

Index segments API

单索引段信息：返回有关索引分片中 Lucene 段的低级信息。

GET /test/_segments

curl -X GET "localhost:9200/test/_segments?pretty"

http://41.1.31.21:39200/_cat/segments/vehicle_pic_index_20200629-20200705?v

多个索引段信息

GET /test1,test2/_segments

curl -X GET "localhost:9200/test1,test2/_segments?pretty"

多个索引段信息

GET /_segments

curl -X GET "localhost:9200/_segments?pretty"

Index recovery API

单索引：返回有关正在进行和已完成的分片恢复的信息。

GET /twitter/_recovery

curl -X GET "localhost:9200/twitter/_recovery?pretty"

获取多个索引的恢复信息

GET index1,index2/_recovery?human

curl -X GET "localhost:9200/index1,index2/_recovery?human&pretty"

获取所有索引的段信息

GET /_recovery?human

curl -X GET "localhost:9200/_recovery?human&pretty"

获取详细的恢复信息

GET _recovery?human&detailed=true

curl -X GET "localhost:9200/_recovery?human&detailed=true&pretty"

Status management

Clear cache API

清除单个索引的缓存

POST /twitter/_cache/clear

curl -X POST "localhost:9200/twitter/_cache/clear?pretty"

清除多个索引的缓存

POST /kimchy,elasticsearch/_cache/clear

curl -X POST "localhost:9200/kimchy,elasticsearch/_cache/clear?pretty"

清除全部索引的缓存

POST /_cache/clear

curl -X POST "localhost:9200/_cache/clear?pretty"

只清除字段缓存

POST /twitter/_cache/clear?fielddata=true

curl -X POST "localhost:9200/twitter/_cache/clear?fielddata=true&pretty"

只清除查询缓存

POST /twitter/_cache/clear?query=true

curl -X POST "localhost:9200/twitter/_cache/clear?query=true&pretty"

只清除请求缓存

POST /twitter/_cache/clear?request=true

curl -X POST "localhost:9200/twitter/_cache/clear?request=true&pretty"

清除特定字段的缓存

POST /twitter/_cache/clear?fields=foo,bar

curl -X POST "localhost:9200/twitter/_cache/clear?fields=foo,bar&pretty"

Refresh API

刷新单个索引

POST /my-index-000001/_refresh

curl -X POST "localhost:9200/my-index-000001/_refresh?pretty"

刷新多个索引

POST /my-index-000001,my-index-000002/_refresh

curl -X POST "localhost:9200/my-index-000001,my-index-000002/_refresh?pretty"

刷新全部索引

POST /_refresh

curl -X POST "localhost:9200/_refresh?pretty"

Flush API

刷新单个索引

POST /my-index-000001/_flush

curl -X POST "localhost:9200/my-index-000001/_flush?pretty"

刷新多个索引

POST /my-index-000001,my-index-000002/_flush

curl -X POST "localhost:9200/my-index-000001,my-index-000002/_flush?pretty"

刷新全部索引

POST /_flush

curl -X POST "localhost:9200/_flush?pretty"

Synced flush API

刷新单个索引

POST /kimchy/_flush/synced

curl -X POST "localhost:9200/kimchy/_flush/synced?pretty"

刷新多个索引

POST /kimchy,elasticsearch/_flush/synced

curl -X POST "localhost:9200/kimchy,elasticsearch/_flush/synced?pretty"

刷新全部索引

POST /_flush/synced

curl -X POST "localhost:9200/_flush/synced?pretty"

Force merge API

强制合并特定索引

POST /twitter/_forcemerge

curl -X POST "localhost:9200/twitter/_forcemerge?pretty"

强制合并多个索引

POST /kimchy,elasticsearch/_forcemerge

curl -X POST "localhost:9200/kimchy,elasticsearch/_forcemerge?pretty"

强制合并所有索引

POST /_forcemerge

curl -X POST "localhost:9200/_forcemerge?pretty"

基于时间的索引
强制合并对于基于时间的索引很有用，尤其是在使用翻转时。在这些情况下，每个索引仅在特定时间段内接收索引流量。一旦索引不再收到写入，它的分片就可以被强制合并到一个段中。

POST /logs-000001/_forcemerge?max_num_segments=1

curl -X POST "localhost:9200/logs-000001/_forcemerge?max_num_segments=1&pretty"

计算公式

分片数

  • 索引实例（当前以时间范围划分）分片数(多个shard会存储在不同服务器上)
  • ES推荐分shard为5-15，故如果计算值<5, 请使用shard为5，但如果数据量特别小，也可使用计算值。
  • 每个分片的数据量20G左右查询最快，最大算30G计算。
shard（分片数）= ES节点数 / (副本数+1)

索引分片数

shard.size(GB) < elasticsearch.jvm.size(GB)
shard.number = index.size(GB) / elasticsearch.jvm.size(GB) + 1

零散

ES启动

  • 方法1
/etc/rc.d/init.d/cluster001-SERVICE-ELASTICSEARCH-retro-1-NODE start
  • 方法2
su - elasticsearch -c '/usr/lib/cluster001/SERVICE-ELASTICSEARCH-retro-1/bin/elasticsearch -d -p /var/run/cluster001/SERVICE-ELASTICSEARCH-retro-1/elasticsearch.pid'

分片分配并发数

PUT /_cluster/settings
{
  "transient": {
    "cluster.routing.allocation.cluster_concurrent_rebalance":15,
    "cluster.routing.allocation.node_concurrent_recoveries":5
  }
}

curl -X PUT "localhost:39200/_cluster/settings?pretty" -H 'Content-Type: application/json' -d'
{
  "transient": {
    "cluster.routing.allocation.cluster_concurrent_rebalance":68,
    "cluster.routing.allocation.node_concurrent_recoveries":2
  }
}
'

persistent 永久 transient 临时

Elasticsearch可以动态设置某些属性，并且可以通过API来进行设置，包括transient和persistent两种方式，

  • transient 临时：这些设置在集群重启之前一直会生效。一旦整个集群重启，这些设置就被清除。
  • persistent 永久：这些设置永久保存，除非再次被手动修改。是将修改持久化到文件中，重启之后也不影响。
PUT /_cluster/settings
{
    "persistent" : {
        "discovery.zen.minimum_master_nodes" : 2  //这个永久设置会在全集群重启时存活下来。
    },
    "transient" : {
        "indices.store.throttle.max_bytes_per_sec" : "50mb"  // 这个临时设置会在第一次全集群重启后被移除。
    }
}

ES查看master

http://41.1.31.21:39200/_cat/master

ES修改max_result_window值

curl -XPUT 'localhost:39200/_settings' -H "Content-Type: application/json" -d '{ "index.max_result_window" :"10000"}'

ES查看节点内存使用情况

curl 'localhost:39200/_cat/nodes?v&h=http,version,jdk,disk.total,disk.used,disk.avail,disk.used_percent,heap.current,heap.percent,heap.max,ram.current,ram.percent,ram.max,master'

ES取消只读

curl -XPUT -H "Content-Type: application/json" http://localhost:39200/_all/_settings -d '{"index.blocks.read_only_allow_delete": false}'

ES修改所有索引的副本数

curl -XPUT -H "Content-Type: application/json" http://127.0.0.1:39200/_all/_settings -d '{"number_of_replicas": "0"}'

ES删除template

curl -X DELETE 'http://21.26.96.204:39200/_template/template_copy'

ES7修改实例的shard数量上限

curl -X PUT 'http://localhost:9200/_cluster/settings' -H "Content-Type: application/json" -d '{ "persistent": { "cluster.max_shards_per_node": "5000" } }'

分片分配并发数

PUT /_cluster/settings
{
  "transient": {
    "cluster.routing.allocation.cluster_concurrent_rebalance":15,
    "cluster.routing.allocation.node_concurrent_recoveries":5
  }
}

curl -X PUT "localhost:39200/_cluster/settings?pretty" -H 'Content-Type: application/json' -d'
{
  "transient": {
    "cluster.routing.allocation.cluster_concurrent_rebalance":68,
    "cluster.routing.allocation.node_concurrent_recoveries":2
  }
}
'

ES重启之前，禁止分片移动

curl -XPUT -H "Content-Type: application/json" http://127.0.0.1:39200/_cluster/settings -d '{
"transient" : {
"cluster.routing.allocation.enable" : "none"
}
}'

ES重启之后恢复分片自动分配

curl -XPUT -H "Content-Type: application/json" http://127.0.0.1:39200/_cluster/settings -d '{
"transient" : {
"cluster.routing.allocation.enable" : null
}
}'

ES分片重新均衡 none 禁用

curl -XPUT -H "Content-Type: application/json" http://127.0.0.1:39200/_cluster/settings -d '{
"transient" : {
"cluster.routing.rebalance.enable" : "none"
}
}'

ES查看还没分配的分片

curl -s 'http://localhost:39200/_cat/shards'|awk '$4 != "STARTED" {print $0}'

ES重新分配失败的分片

curl -XPOST 'http://localhost:9200/_cluster/reroute?retry_failed=true'

ES清空fielddata缓存

curl -XPOST 'http://localhost:9200/_cache/clear?fielddata=true'

ES修改副本数

PUT /_all/_settings
{
  "number_of_replicas": 1
}

#节点退役(除了_ip 之外, 还可以用_name、_host进行匹配)
PUT /_cluster/settings
{
  "transient": {
    "cluster.routing.allocation.exclude._ip":"33.95.253.143"
  }
}

ES迁移shard

curl -X POST "localhost:9200/_cluster/reroute?pretty" -H 'Content-Type: application/json' -d'
{
  "commands": [
    {
      "move": {
        "index": "test", "shard": 0,
        "from_node": "node1", "to_node": "node2"
      }
    },
    {
      "allocate_replica": {
        "index": "test", "shard": 1,
        "node": "node3"
      }
    }
  ]
}
'

ES段合并

POST /vehicle_pic_index_20210517-20210523/_forcemerge?max_num_segments=1

ES优化参数

indices.requests.cache.size: 2%
indices.queries.cache.size: 2%
indices.fielddata.cache.size: 40%

写入前

curl -XPUT -H "Content-Type: application/json" http://127.0.0.1:39200/zs0612/_settings -d '{
"refresh_interval": "-1",
"number_of_replicas": "0",
"index.translog.durability": "async",
"index.translog.sync_interval": "60s"
}'

写入后

curl -XPUT -H "Content-Type: application/json" http://127.0.0.1:39200/zs0612/_settings -d '{
"number_of_replicas": "1",
"refresh_interval": null,
"index.translog.durability": null,
"index.translog.sync_interval": null
}'

插件管理

插件位置指定

获取插件命令使用说明

sudo bin /elasticsearch - plugin -h

指定URL直接从自定义位置下载插件

sudo bin /elasticsearch - plugin install [url]

UNIX环境：插件位置指定

sudo bin /elasticsearch - plugin install file: ///path/to/plugin.zip

windows环境：插件位置指定

bin /elasticsearch - plugin install file: ///C:/path/to/plugin.zip

HTTP：插件位置指定

sudo bin /elasticsearch - plugin install http: //path/to/plugin.zip

安装插件

sudo bin /elasticsearch - plugin install [plugin_name]

列出当前插件

sudo bin /elasticsearch - plugin list

删除插件

sudo bin /elasticsearch - remove [plugin_name]

更新插件

插件是为特定版本的elasticsearch构建的，因此每次更新都必须重新安装插件

sudo bin /elasticsearch - remove [plugin_name]
sudo bin /elasticsearch - plugin install [plugin_name]

原创文章，作者：kirin，如若转载，请注明出处：https://blog.ytso.com/tech/pnotes/274223.html

赞 (0)
0 0
上一篇 2022年7月14日 02:07
下一篇 2022年7月14日 02:07

相关推荐

发表回复

登录后才能评论