Elasticsearch中在node的配置中可以指定path.data用来作为节点数据的存储目录而且我们可以指定多个值来作为数据存储的路径那么Elasticsearch是如何判断应该存储到哪个路径下呢今天我就记录一下这个问题。Elasticsearch的索引创建过程集群master收到创建索引的请求后经过创建索引的一些步骤最终会将索引创建完成的请求提交到ClusterStatemaster将根据ClusterState分发给所有节点涉及创建shard的节点会读取本地可用的path.data然后依据一定的规则获取路径。创建基本shard路径保存基本的shard信息。如何确定在哪个目录下源码主要调用的是ShardPath的selectNewPathForShard方法for (NodeEnvironment.NodePath nodePath : env.nodePaths()) { totFreeSpace totFreeSpace.add(BigInteger.valueOf(nodePath.fileStore.getUsableSpace())); } // TODO: this is a hack!! We should instead keep track of incoming (relocated) shards since we know // how large they will be once theyre done copying, instead of a silly guess for such cases: // Very rough heuristic of how much disk space we expect the shard will use over its lifetime, the max of current average // shard size across the cluster and 5% of the total available free space on this node: BigInteger estShardSizeInBytes BigInteger.valueOf(avgShardSizeInBytes).max(totFreeSpace.divide(BigInteger.valueOf(20))); // TODO - do we need something more extensible? Yet, this does the job for now... final NodeEnvironment.NodePath[] paths env.nodePaths(); // If no better path is chosen, use the one with the most space by default NodeEnvironment.NodePath bestPath getPathWithMostFreeSpace(env); if (paths.length ! 1) { MapNodeEnvironment.NodePath, Long pathToShardCount env.shardCountPerPath(shardId.getIndex()); // Compute how much space there is on each path final MapNodeEnvironment.NodePath, BigInteger pathsToSpace new HashMap(paths.length); for (NodeEnvironment.NodePath nodePath : paths) { FileStore fileStore nodePath.fileStore; BigInteger usableBytes BigInteger.valueOf(fileStore.getUsableSpace()); pathsToSpace.put(nodePath, usableBytes); } bestPath Arrays.stream(paths) // Filter out paths that have enough space .filter((path) - pathsToSpace.get(path).subtract(estShardSizeInBytes).compareTo(BigInteger.ZERO) 0) // Sort by the number of shards for this index .sorted((p1, p2) - { int cmp Long.compare(pathToShardCount.getOrDefault(p1, 0L), pathToShardCount.getOrDefault(p2, 0L)); if (cmp 0) { // if the number of shards is equal, tie-break with the number of total shards cmp Integer.compare(dataPathToShardCount.getOrDefault(p1.path, 0), dataPathToShardCount.getOrDefault(p2.path, 0)); if (cmp 0) { // if the number of shards is equal, tie-break with the usable bytes cmp pathsToSpace.get(p2).compareTo(pathsToSpace.get(p1)); } } return cmp; }) // Return the first result .findFirst() // Or the existing best path if there arent any that fit the criteria .orElse(bestPath); } statePath bestPath.resolve(shardId); dataPath statePath; }过程分析首先判断是否自定义了path.data没有自定义就在默认路径下创建自定义的情况下确保节点下最少有5%的空间可以使用获取所有的paths然后设置默认最佳的path是当前拥有最多空间的path遍历所有的paths首先过滤掉没有空间的path如果最终没有符合的就返回4步骤的path否则继续6步骤按照规则对paths排序首先判断每个path下该索引的shard数优先返回含有本索引的shard数最少的path当条件结果相同对比每个path中包含有的shard总数所有索引的返回包含shard数最少的path当2条件结果相同对比可用空间返回可用空间最大的path生成相应的路径创建目录等信息。
Elasticsearch索引的数据存储路径是如何确定的
Elasticsearch中在node的配置中可以指定path.data用来作为节点数据的存储目录而且我们可以指定多个值来作为数据存储的路径那么Elasticsearch是如何判断应该存储到哪个路径下呢今天我就记录一下这个问题。Elasticsearch的索引创建过程集群master收到创建索引的请求后经过创建索引的一些步骤最终会将索引创建完成的请求提交到ClusterStatemaster将根据ClusterState分发给所有节点涉及创建shard的节点会读取本地可用的path.data然后依据一定的规则获取路径。创建基本shard路径保存基本的shard信息。如何确定在哪个目录下源码主要调用的是ShardPath的selectNewPathForShard方法for (NodeEnvironment.NodePath nodePath : env.nodePaths()) { totFreeSpace totFreeSpace.add(BigInteger.valueOf(nodePath.fileStore.getUsableSpace())); } // TODO: this is a hack!! We should instead keep track of incoming (relocated) shards since we know // how large they will be once theyre done copying, instead of a silly guess for such cases: // Very rough heuristic of how much disk space we expect the shard will use over its lifetime, the max of current average // shard size across the cluster and 5% of the total available free space on this node: BigInteger estShardSizeInBytes BigInteger.valueOf(avgShardSizeInBytes).max(totFreeSpace.divide(BigInteger.valueOf(20))); // TODO - do we need something more extensible? Yet, this does the job for now... final NodeEnvironment.NodePath[] paths env.nodePaths(); // If no better path is chosen, use the one with the most space by default NodeEnvironment.NodePath bestPath getPathWithMostFreeSpace(env); if (paths.length ! 1) { MapNodeEnvironment.NodePath, Long pathToShardCount env.shardCountPerPath(shardId.getIndex()); // Compute how much space there is on each path final MapNodeEnvironment.NodePath, BigInteger pathsToSpace new HashMap(paths.length); for (NodeEnvironment.NodePath nodePath : paths) { FileStore fileStore nodePath.fileStore; BigInteger usableBytes BigInteger.valueOf(fileStore.getUsableSpace()); pathsToSpace.put(nodePath, usableBytes); } bestPath Arrays.stream(paths) // Filter out paths that have enough space .filter((path) - pathsToSpace.get(path).subtract(estShardSizeInBytes).compareTo(BigInteger.ZERO) 0) // Sort by the number of shards for this index .sorted((p1, p2) - { int cmp Long.compare(pathToShardCount.getOrDefault(p1, 0L), pathToShardCount.getOrDefault(p2, 0L)); if (cmp 0) { // if the number of shards is equal, tie-break with the number of total shards cmp Integer.compare(dataPathToShardCount.getOrDefault(p1.path, 0), dataPathToShardCount.getOrDefault(p2.path, 0)); if (cmp 0) { // if the number of shards is equal, tie-break with the usable bytes cmp pathsToSpace.get(p2).compareTo(pathsToSpace.get(p1)); } } return cmp; }) // Return the first result .findFirst() // Or the existing best path if there arent any that fit the criteria .orElse(bestPath); } statePath bestPath.resolve(shardId); dataPath statePath; }过程分析首先判断是否自定义了path.data没有自定义就在默认路径下创建自定义的情况下确保节点下最少有5%的空间可以使用获取所有的paths然后设置默认最佳的path是当前拥有最多空间的path遍历所有的paths首先过滤掉没有空间的path如果最终没有符合的就返回4步骤的path否则继续6步骤按照规则对paths排序首先判断每个path下该索引的shard数优先返回含有本索引的shard数最少的path当条件结果相同对比每个path中包含有的shard总数所有索引的返回包含shard数最少的path当2条件结果相同对比可用空间返回可用空间最大的path生成相应的路径创建目录等信息。
