一篇就懂 Elasticsearch | 原力计划
-
实时性ES高于Solr -
Solr传统搜索性能高于ES
-
去官网下载获得压缩包一个，版本6.2.4 -
解压缩进入目录找到bin文件夹 -
双击运行bin文件夹中elasticsearch.bat脚本 -
在浏览器输入http://localhost:9200/
1
useradd 用户名
1
passwd 用户名
1
su 用户名
1
tar
-zxvf
elasticsearch-6
.2.4.tar.gz
1
mv elasticsearch-6.2.4/ elasticsearch
1
ls -l /home/用户名/elasticsearch
1
vim
jvm
.options
1
-Xms1g
2
-Xmx1g
1
-Xms512m
2
-Xmx512m
1
vim
elasticsearch
.yml
-
修改数据和日志目录：
1path.data:
/home/用户名/elasticsearch/data
# 数据目录位置
2path.logs:
/home/用户名/elasticsearch/logs
# 日志目录位置
1
mkdir data
2
mkdir logs
-
修改绑定的ip：
1
network
.host: 0
.0.0.0 # 绑定到0
.0.0.0，允许任何
ip来访问
1./elasticsearch
-
错误1： 内核过低
1
bootstrap.sy
stem_call_filter: false
-
错误2： 文件权限不足
1
[1]:
max
file
descriptors
[4096]
for
elasticsearch
process
likely
too
low,
increase
to
at
least
[65536]
1
vim /etc/security/limits.conf
1
* soft nofile 65536
2
3
* hard nofile 131072
4
5
* soft nproc 4096
6
7
* hard nproc 4096
-
错误3： 线程数不够
1
[1]:
max
number
of
threads
[1024]
for
user
[leyou]
is
too
low,
increase
to
at
least
[4096]
1
vim /etc/security/limits.d/
90-nproc.conf
1
* soft nproc 1024
1
* soft nproc 4096
-
错误4： 进程虚拟内存
1
[3]: max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]
1
vim /etc/sysctl.conf
1vm.max_map_count=655360
1
sysctl -p
1./elasticsearch
-
9300： 集群节点间通讯接口 -
9200： 客户端访问接口
1Mysql -> database -> table -> rows -> columns
2
3Elasticsearch -> index -> type -> documents -> fields
1PUT /索引名
2
3{
4
5
"mappings": {
6
7
"type名": {
8
9
"properties": {
10
11
"fields名": {
12
13
"type":
"类型",
14
15
"store": 是否储存[
true,
false],
16
17
"index": 是否索引[
true,
false],
18
19
"analyzer":
"分词器"
20
21 },
22
23
"fields名": {
24
25
"type":
"text",
26
27
"store":
true,
28
29
"index":
true,
30
31
"analyzer":
"standard"
32
33 }
34
35 }
36
37 }
38
39 }
40
41}
1POST /索引名/
type名/_mapping
2
3{
4
5
"type名":{
6
7
"properties":{
8
9
"fields名":{
10
11
"type":
"long",
12
13
"store":
true,
14
15
"index":
false
16
17 },
18
19
"fields名":{
20
21
"type":
"text",
22
23
"store":
true,
24
25
"index":
true,
26
27
"analyzer":
"standard"
28
29 }
30
31 }
32
33 }
34
35}
1
DELETE /索引名
1POST /索引名/
type名/[_id]
2
3{
4
5
"FIELD":
"VALUE",
6
7
"FIELD":
"VALUE",
8
9
"FIELD":
"VALUE"
10
11}
1和添加相同，并且_
id存在
1
DELETE /索引名/type名/_id
1GET /索引名/
type名/_id
1POST /索引名/
type名/_search
2
3{
4
5
"query": {
6
7
"term": {
8
9
"FIELD":
"VALUE"
10
11 }
12
13 }
14
15}
1POST /索引名/
type名/_search
2
3{
4
5
"query": {
6
7
"query_string": {
8
9
"default_field":
"FIELD",
10
11
"query":
"this AND that OR thus"
12
13 }
14
15 }
16
17}
1
GET /_analyze
2
3{
4
5
"analyzer":
"standard",
6
7
"text":
"VALUE"
8
9}
-
下载IK分词器压缩包并解压缩 -
把其中elasticsearch文件夹改名ik-analyzer -
把 ik -analyzer文件夹放置在elasticsearch目录的plugins文件夹 -
重启e lasticsearch服务器 -
测 试分词器效果，把analyzer的值改成ik_smart 或 ik_max_word
1#节点
n的配置信息：
2
3#集群名称，保证唯一
4
5
cluster
.name:
elasticsearch
6
7#节点名称，必须不一样
8
9
node
.name:
node-1
10
11#必须为本机的
ip地址
12
13
network
.host: 127
.0
.0
.1
14
15#服务端口号，在同一机器下必须不一样
16
17
http
.port: 9200
18
19#集群间通信端口号，在同一机器下必须不一样
20
21
transport
.tcp
.port: 9300
22
23#设置集群自动发现机器
ip集合
24
25
discovery
.zen
.ping
.unicast
.hosts:
["127.0.0.1:9300","127.0.0.1:9301","127.0.0.1:9302"]
1
<dependencies>
2
3
<dependency>
4
5
<groupId>org.elasticsearch
</groupId>
6
7
<artifactId>elasticsearch
</artifactId>
8
9
<version>5.6.10
</version>
10
11
</dependency>
12
13
<dependency>
14
15
<groupId>org.elasticsearch.client
</groupId>
16
17
<artifactId>transport
</artifactId>
18
19
<version>5.6.10
</version>
20
21
</dependency>
22
23
<dependency>
24
25
<groupId>org.apache.logging.log4j
</groupId>
26
27
<artifactId>log4j-to-slf4j
</artifactId>
28
29
<version>2.9.1
</version>
30
31
</dependency>
32
33
<dependency>
34
35
<groupId>org.slf4j
</groupId>
36
37
<artifactId>slf4j-api
</artifactId>
38
39
<version>1.7.24
</version>
40
41
</dependency>
42
43
<dependency>
44
45
<groupId>org.slf4j
</groupId>
46
47
<artifactId>slf4j-simple
</artifactId>
48
49
<version>1.7.21
</version>
50
51
</dependency>
52
53
<dependency>
54
55
<groupId>log4j
</groupId>
56
57
<artifactId>log4j
</artifactId>
58
59
<version>1.2.12
</version>
60
61
</dependency>
62
63
<dependency>
64
65
<groupId>junit
</groupId>
66
67
<artifactId>junit
</artifactId>
68
69
<version>4.12
</version>
70
71
</dependency>
72
73
</dependencies>
1
public
class ElasticsearchClient {
2
3
@Test
4
5
public void createIndex() throws Exception {
6
7
// 创建settings对象
8
9 Settings settings = Settings.builder()
10
11 .put(
"cluster.name",
"elasticsearch")
12
13 .build();
14
15
// 创建客户端对象
16
17 TransportClient client =
new PreBuiltTransportClient(settings);
18
19 client.addTransportAddress(
new InetSocketTransportAddress(InetAddress.getByName(
"127.0.0.1"),
9300));
20
21
// 创建索引库
22
23 client.admin().indices().prepareCreate(
"index_hello").get();
24
25
// 关闭资源
26
27 client.close();
28
29 }
30
31}
1
public
class ElasticsearchClient {
2
3
@Test
4
5
public void setMapping() throws Exception {
6
7
// 创建settings对象
8
9 Settings settings = Settings.builder()
10
11 .put(
"cluster.name",
"elasticsearch")
12
13 .build();
14
15
// 创建客户端对象
16
17 TransportClient client =
new PreBuiltTransportClient(settings);
18
19 client.addTransportAddress(
new InetSocketTransportAddress(InetAddress.getByName(
"127.0.0.1"),
9300));
20
21
// 构建Mapping,RESTfull请求体
22
23 XContentBuilder builder = XContentFactory.jsonBuilder()
24
25 .startObject()
26
27 .startObject(
"article")
28
29 .startObject(
"properties")
30
31 .startObject(
"id")
32
33 .field(
"type",
"long")
34
35 .field(
"store",
true)
36
37 .endObject()
38
39 .startObject(
"title")
40
41 .field(
"type",
"text")
42
43 .field(
"store",
true)
44
45 .field(
"analyzer",
"ik_smart")
46
47 .endObject()
48
49 .startObject(
"content")
50
51 .field(
"type",
"text")
52
53 .field(
"store",
true)
54
55 .field(
"analyzer",
"ik_smart")
56
57 .endObject()
58
59 .endObject()
60
61 .endObject()
62
63 .endObject();
64
65
// 使用客户端把mapping设置到索引库
66
67 client.admin().indices()
68
69 .preparePutMapping(
"index_hello")
70
71 .setType(
"article")
72
73 .setSource(builder)
74
75 .get();
76
77
// 关闭资源
78
79 client.close();
80
81 }
82
83}
1
public
class ElasticsearchClient {
2
3
@Test
4
5
public void addDocument() throws Exception {
6
7
// 创建settings对象
8
9 Settings settings = Settings.builder()
10
11 .put(
"cluster.name",
"elasticsearch")
12
13 .build();
14
15
// 创建客户端对象
16
17 TransportClient client =
new PreBuiltTransportClient(settings);
18
19 client.addTransportAddress(
new InetSocketTransportAddress(InetAddress.getByName(
"127.0.0.1"),
9300));
20
21 XContentBuilder builder = XContentFactory.jsonBuilder()
22
23 .startObject()
24
25 .field(
"id",
1L)
26
27 .field(
"title",
"商务部：有序有力组织商贸企业复工复产")
28
29 .field(
"content",
"新华社北京2月23日电（记者陈炜伟、王雨萧）记者23日从商务部了解到，商务部印发《关于统筹做好生活必需品供应保障有关工作的通知》，要求各地商务主管部门在确保疫情防控安全的前提下，有序有力组织商贸企业复工复产。")
30
31 .endObject();
32
33
// 发送到服务器
34
35 client.prepareIndex(
"index_hello",
"article",
"1")
36
37 .setSource(builder)
38
39 .get();
40
41
// 关闭资源
42
43 client.close();
44
45
46
47 }
48
49}
-
根据_Id查询 -
根据Term查询 -
根 据QueryString查询 -
设置分页（在执行查询之前SearchResponse中方法setFrom和setSize） -
设置高亮
1
public
class
SearchIndex {
2
3
private TransportClient client;
4
5 @
Before
6
7 public void init()throws Exception {
8
9
// 创建settings对象
10
11 Settings settings = Settings.builder()
12
13 .put(
"cluster.name",
"elasticsearch")
14
15 .build();
16
17
// 创建客户端对象
18
19 client =
new PreBuiltTransportClient(settings);
20
21 client.addTransportAddress(
new InetSocketTransportAddress(InetAddress.getByName(
"127.0.0.1"),
9300));
22
23 }
24
25 @
Test
26
27 public void searchById(){
28
29
// 创建查询对象
30
31 QueryBuilder queryBuilder = QueryBuilders.idsQuery().addIds(
"1",
"2");
32
33
// 执行查询
34
35 SearchResponse searchResponse = client.prepareSearch(
"index_hello")
36
37 .setTypes(
"article")
38
39 .setQuery(queryBuilder)
40
41 .
get();
42
43
// 获取总记录数
44
45 SearchHits hits = searchResponse.getHits();
46
47 System.
out.println(
"总记录数："+hits.getTotalHits());
48
49
// 结果列表
50
51 Iterator<SearchHit> iterator = hits.iterator();
52
53
while (iterator.hasNext()){
54
55 SearchHit next = iterator.next();
56
57 Map<String, Object> source = next.getSource();
58
59 System.
out.println(
"id:"+source.
get(
"id"));
60
61 System.
out.println(
"title:"+source.
get(
"title"));
62
63 System.
out.println(
"content:"+source.
get(
"content"));
64
65 System.
out.println(
"----------------------------------");
66
67 }
68
69
70
71 }
72
73 @
Test
74
75 public void searchByTerm(){
76
77
// 创建查询对象
78
79 QueryBuilder queryBuilder = QueryBuilders.termQuery(
"title",
"复工");
80
81
// 执行查询
82
83 SearchResponse searchResponse = client.prepareSearch(
"index_hello")
84
85 .setTypes(
"article")
86
87 .setQuery(queryBuilder)
88
89 .
get();
90
91
// 获取总记录数
92
93 SearchHits hits = searchResponse.getHits();
94
95 System.
out.println(
"总记录数："+hits.getTotalHits());
96
97
// 结果列表
98
99 Iterator<SearchHit> iterator = hits.iterator();
100
101
while (iterator.hasNext()){
102
103 SearchHit next = iterator.next();
104
105 Map<String, Object> source = next.getSource();
106
107 System.
out.println(
"id:"+source.
get(
"id"));
108
109 System.
out.println(
"title:"+source.
get(
"title"));
110
111 System.
out.println(
"content:"+source.
get(
"content"));
112
113 System.
out.println(
"----------------------------------");
114
115 }
116
117
118
119 }
120
121 @
Test
122
123 public void searchByQueryString(){
124
125
// 创建查询对象
126
127 QueryBuilder queryBuilder = QueryBuilders.queryStringQuery(
"我家住在临安").defaultField(
"title");
128
129
// 执行查询
130
131 SearchResponse searchResponse = client.prepareSearch(
"index_hello")
132
133 .setTypes(
"article")
134
135 .setQuery(queryBuilder)
136
137 .setFrom(
0)
138
139 .setSize(
3)
140
141 .highlighter(
new HighlightBuilder().field(
"title").preTags(
"<em>").postTags(
"</em>"))
142
143 .
get();
144
145
// 获取总记录数
146
147 SearchHits hits = searchResponse.getHits();
148
149 System.
out.println(
"总记录数："+hits.getTotalHits());
150
151
// 结果列表
152
153 Iterator<SearchHit> iterator = hits.iterator();
154
155
while (iterator.hasNext()){
156
157 SearchHit next = iterator.next();
158
159 Map<String, Object> source = next.getSource();
160
161 System.
out.println(
"id:"+source.
get(
"id"));
162
163 System.
out.println(
"title:"+source.
get(
"title"));
164
165 System.
out.println(
"content:"+source.
get(
"content"));
166
167 System.
out.println(
"----------------------------------");
168
169 HighlightField title = next.getHighlightFields().
get(
"title");
170
171 System.
out.println(title.getFragments()[
0]);
172
173 }
174
175
176
177 }
178
179 @
After
180
181 public void close(){
182
183 client.close();
184
185 }
186
187}
1
<dependencies>
2
3
<dependency>
4
5
<groupId>org.elasticsearch
</groupId>
6
7
<artifactId>elasticsearch
</artifactId>
8
9
<version>5.6.10
</version>
10
11
</dependency>
12
13
<dependency>
14
15
<groupId>org.elasticsearch.client
</groupId>
16
17
<artifactId>transport
</artifactId>
18
19
<version>5.6.10
</version>
20
21
</dependency>
22
23
<dependency>
24
25
<groupId>org.apache.logging.log4j
</groupId>
26
27
<artifactId>log4j-to-slf4j
</artifactId>
28
29
<version>2.9.1
</version>
30
31
</dependency>
32
33
<dependency>
34
35
<groupId>org.slf4j
</groupId>
36
37
<artifactId>slf4j-api
</artifactId>
38
39
<version>1.7.24
</version>
40
41
</dependency>
42
43
<dependency>
44
45
<groupId>org.slf4j
</groupId>
46
47
<artifactId>slf4j-simple
</artifactId>
48
49
<version>1.7.21
</version>
50
51
</dependency>
52
53
<dependency>
54
55
<groupId>log4j
</groupId>
56
57
<artifactId>log4j
</artifactId>
58
59
<version>1.2.12
</version>
60
61
</dependency>
62
63
<dependency>
64
65
<groupId>junit
</groupId>
66
67
<artifactId>junit
</artifactId>
68
69
<version>4.12
</version>
70
71
</dependency>
72
73
<dependency>
74
75
<groupId>com.fasterxml.jackson.core
</groupId>
76
77
<artifactId>jackson-core
</artifactId>
78
79
<version>2.8.1
</version>
80
81
</dependency>
82
83
<dependency>
84
85
<groupId>com.fasterxml.jackson.core
</groupId>
86
87
<artifactId>jackson-databind
</artifactId>
88
89
<version>2.8.1
</version>
90
91
</dependency>
92
93
<dependency>
94
95
<groupId>com.fasterxml.jackson.core
</groupId>
96
97
<artifactId>jackson-annotations
</artifactId>
98
99
<version>2.8.1
</version>
100
101
</dependency>
102
103
<dependency>
104
105
<groupId>org.springframework.data
</groupId>
106
107
<artifactId>spring-data-elasticsearch
</artifactId>
108
109
<version>3.0.9.RELEASE
</version>
110
111
<exclusions>
112
113
<exclusion>
114
115
<groupId>org.elasticsearch.plugin
</groupId>
116
117
<artifactId>transport-netty4-client
</artifactId>
118
119
</exclusion>
120
121
</exclusions>
122
123
</dependency>
124
125
<dependency>
126
127
<groupId>org.springframework
</groupId>
128
129
<artifactId>spring-test
</artifactId>
130
131
<version>5.0.8.RELEASE
</version>
132
133
<scope>test
</scope>
134
135
</dependency>
136
137
138
139
</dependencies>
1
<?xml version="1.0" encoding="UTF-8"?>
2
3
<beans xmlns="http://www.springframework.org/schema/beans"
4
5 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
6
7 xmlns:context="http://www.springframework.org/schema/context"
8
9 xmlns:elasticsearch="http://www.springframework.org/schema/data/elasticsearch"
10
11 xsi:schemaLocation="
12
13http://www.springframework.org/schema/beans
14
15http://www.springframework.org/schema/beans/spring-beans.xsd
16
17http://www.springframework.org/schema/context
18
19http://www.springframework.org/schema/context/spring-context.xsd
20
21http://www.springframework.org/schema/data/elasticsearch
22
23http://www.springframework.org/schema/data/elasticsearch/spring-elasticsearch-1.0.xsd
24
25">
26
27
<!-- 客户端对象 -->
28
29
<elasticsearch:transport-client id="esClient" cluster-name="elasticsearch" cluster-nodes="127.0.0.1:9300"/>
30
31
<!-- 包扫描器 -->
32
33
<elasticsearch:repositories base-package="com.itheima.es.repositories"/>
34
35
<!-- 模板对象 -->
36
37
<bean id="elasticsearchTemplate" class="org.springframework.data.elasticsearch.core.ElasticsearchTemplate">
38
39
<constructor-arg name="client" ref="esClient"/>
40
41
</bean>
42
43
</beans>
1
@Document(indexName = "my_blog",type = "article")
2
3
public
class Article {
4
5
@Id
6
7
@Field(type = FieldType.Long)
8
9
private
Long id;
10
11
@Field(type = FieldType.Text,analyzer = "ik_smart")
12
13
private String title;
14
15
@Field(type = FieldType.Text,analyzer = "ik_smart")
16
17
private String content;
18
19
20
21
public Article() {
22
23 }
24
25
26
27
public Article(
Long id, String title, String content) {
28
29
this.id = id;
30
31
this.title = title;
32
33
this.content = content;
34
35 }
36
37
38
39
public
Long getId() {
40
41
return id;
42
43 }
44
45
46
47
public void setId(
Long id) {
48
49
this.id = id;
50
51 }
52
53
54
55
public String getTitle() {
56
57
return title;
58
59 }
60
61
62
63
public void setTitle(String title) {
64
65
this.title = title;
66
67 }
68
69
70
71
public String getContent() {
72
73
return content;
74
75 }
76
77
78
79
public void setContent(String content) {
80
81
this.content = content;
82
83 }
84
85
86
87
@Override
88
89
public String toString() {
90
91
return
"Article{" +
92
93
"id=" + id +
94
95
", title='" + title +
'\'' +
96
97
", content='" + content +
'\'' +
98
99
'}';
100
101 }
102
103}
1
public
interface ArticleRepository extends ElasticsearchRepository<Article,Long> {
2
3}
1
@RunWith(SpringRunner.class)
2
3
@ContextConfiguration(
"classpath:applicationContext.xml")
4
5
public
class ESTest {
6
7
@Autowired
8
9
private ArticleRepository articleRepository;
10
11
@Autowired
12
13
private ElasticsearchTemplate template;
14
15
@Test
16
17
public void createIndex(){
18
19
// 创建索引并配置映射关系
20
21 template.createIndex(Article.class);
22
23
// 配置映射
24
25
// template.putMapping(Article.class);
26
27 }
28
29
@Test
30
31
public void addDocumentAndUpdateDocument(){
32
33 Article article =
new Article(
2L,
"时政新闻眼丨在一场罕见的电视电话会上，习近平这样动员战“疫”",
"2月23日，农历二月初一。一场特别的会议在人民大会堂召开，分会场一直设到了县、团。中国正在打一场疫情防控的人民战争、总体战、阻击战。怎样分析这场战争？目前打到了哪一步？如何全面打赢？亲自指挥这场战争的习近平在这场“战时会议”上从容作答。");
34
35
// 添加文档
36
37 articleRepository.save(article);
38
39 }
40
41
@Test
42
43
public void delDocument(){
44
45
// 根据Id删除
46
47 articleRepository.deleteById(
1L);
48
49
// 全部删除
50
51
// articleRepository.deleteAll();
52
53 }
54
55
@Test
56
57
public void findAll(){
58
59 Iterable<Article> all = articleRepository.findAll();
60
61 all.forEach(a-> System.out.println(a));
62
63 }
64
65
@Test
66
67
public void findById(){
68
69 Optional<Article> optional = articleRepository.findById(
2L);
70
71 System.out.println(optional.get());
72
73 }
74
75}
1 @Test
2
3
public void nativeSearch(){
4
5 NativeSearchQuery query =
new NativeSearchQueryBuilder()
6
7 .withQuery(
8
9 QueryBuilders.queryStringQuery(
"测试一个查询")
10
11 .defaultField(
"title")
12
13 ).withPageable(PageRequest.of(
0,
15))
14
15 .build();
16
17 AggregatedPage<Article> articles =
template.queryForPage(query, Article.class);
18
19 articles.forEach(a-> System.out.println(a));
20
21 }
-
String类型，又分两种： -
text： 可分词，不可参与聚合 -
keyword： 不可分词，数据会作为完整字段进行匹配，可以参与聚合
-
Numerical： 数值类型，分两类 -
基本数据类型： long、interger、short、byte、double、float、half_float -
浮点数的高精度类型： scaled_float -
需要指定一个精度因子，比如10或100。 elasticsearch会把真实值乘以这个因子后存储，取出时再还原。
-
Date： 日期类型
-
true： 字段会被索引，则可以用来进行搜索。 默认值就是true -
false： 字段不会被索引，不能用来搜索