Created: May 24, 2022 11:14 AM Tags: 현대차증권 분류: 솔루션설치
모니터링 서버 설치
0. 설치순서
- openjdk 설치
- Elasticsearch 설치
- Kibana 설치
- 각 node 에 logstash 설치
- 로그 확인 / data 확인
1. openjdk설치
- 설치
rpm -Uvh copy-jdk-configs-4.0-2.el8.noarch.rpm \
&& rpm -Uvh javapackages-filesystem-5.3.0-1.module_el8.0.0+11+5b8c10bd.noarch.rpm \
&& rpm -Uvh lksctp-tools-1.0.18-3.el8.x86_64.rpm \
&& rpm -Uvh tzdata-java-2021e-1.el8.noarch.rpm \
&& rpm -Uvh java-11-openjdk-headless-11.0.13.0.8-4.el8_5.x86_64.rpm
- 그 외 package 설치 → 그 외 기본값으로 설치되어 있음 → rqm -pa로 확인만 진행
rpm -qa | grep 'alsa-lib-\|avahi-libs-\|crypto-policies-\|cups-libs\|freetype-\|graphite2-\|harfbuzz-\|lcms2-\|libjpeg\|libpng\|lua-\|nspr-\|nss-'
result
alsa-lib-1.2.5-4.el8.x86_64.rpm
avahi-libs-0.7-20.el8.x86_64.rpm
crypto-policies-20210617-1.gitc776d3e.el8.noarch.rpm
crypto-policies-scripts-20210617-1.gitc776d3e.el8.noarch.rpm
cups-libs-2.2.6-40.el8.x86_64.rpm
freetype-2.9.1-4.el8_3.1.x86_64.rpm
graphite2-1.3.10-10.el8.x86_64.rpm
harfbuzz-1.7.5-3.el8.x86_64.rpm
lcms2-2.9-2.el8.x86_64.rpm
libjpeg-turbo-1.5.3-12.el8.x86_64.rpm
libpng-1.6.34-5.el8.x86_64.rpm
lua-5.3.4-12.el8.x86_64.rpm
lua-libs-5.3.4-12.el8.x86_64.rpm
nspr-4.32.0-1.el8_4.x86_64.rpm
nss-3.67.0-7.el8_5.x86_64.rpm
nss-softokn-3.67.0-7.el8_5.x86_64.rpm
nss-softokn-freebl-3.67.0-7.el8_5.x86_64.rpm
nss-sysinit-3.67.0-7.el8_5.x86_64.rpm
nss-util-3.67.0-7.el8_5.x86_64.rpm
2. Elasticsearch 설치
sudo dnf install --disablerepo=* -y elasticsearch-oss
sudo systemctl daemon-reload
sudo systemctl enable --now elasticsearch.service
sudo systemctl status elasticsearch
sudo systemctl stop elasticsearch.service
elasticsearch.yml설정 변경
vi /etc/elasticsearch/elasticsearch.yml
path.data= /mceadm/apps/elasticsearch/data
path.logs= /mceadm/apps/elasticsearch/log
network.host: 10.227.108.249
http.port: 9200
discovery.seed_hosts : ["10.227.108.249"]
cluster.initial_master_nodes : ["10.227.171.99"]
**@data, log directory 변경**
usermod -a -G hmc elasticsearch
mkdir -p /mceadm/apps/elasticsearch/data
mkdir -p /mceadm/apps/elasticsearch/log
chown mceadm:hmc /mceadm/apps/elasticsearch/
chown mceadm:hmc /mceadm/apps/
chown elasticsearch:elasticsearch /mceadm/apps/elasticsearch/data/
chown elasticsearch:elasticsearch /mceadm/apps/elasticsearch/log/
※ 참고 : one node 라도 cluster 설정이 필요 : https://stackoverflow.com/questions/59350069/elasticsearch-start-up-error-the-default-discovery-settings-are-unsuitable-for#59840175
Elasticsearch index확인
curl -XGET '10.227.171.99:9200/_cat/indices?v&pretty'
3. kibana 설치
localhost:5601에서 확인
sudo dnf install --disablerepo=* -y kibana-oss
systemctl enable kibana
systemctl start kibana
※ 참고 : elasticsearch 와 연결이 안되어 있어도 status 는 정상으로 보인다. 다만 log 에 ‘연결이 안되었다’고 보여준다.
kibana.yml수정
vi /etc/kibana/kibana.yml
server.host: "10.227.108.249"
server.port: 5601
elasticsearch.hosts: ["http://10.227.108.249:9200"]
키바나 재시작
systemctl restart kibana
※ 참고 : elasticsearch 가 실행된 상태에서 재시작 되어야 정상작동한다. -*실행이 안되어 있어도 kibana 실행은 되지만 정상작동하지 않음
확인
curl http://10.227.108.249:5601/api/status
curl -vL http://10.227.108.249:5601
4. logstash 설치
설치
dnf install --disablerepo=* -y ./logstash-oss-7.10.2-x86_64.rpm
systemctl enable logstash
cp ./logstash.conf /etc/logstash/conf.d
chmod 644 /etc/logstash/conf.d/logstash.conf
systemctl start logstash
logstash.conf수정
# 개발쪽에선 mongodb port 와 elasticsearch 주소를 수정하자.
input {
# https://www.elastic.co/guide/en/logstash/current/plugins-inputs-exec.html
exec {
command => "mongo mongodb://127.0.0.1:27617/admin -u root -p hmsecroot1@ --authenticationDatabase admin --eval 'db.serverStatus()' --quiet | sed 's/\(NumberLong([[:punct:]]\?\)\([[:digit:]]*\)\([[:punct:]]\?)\)/\2/' | sed 's/\(ISODate(\)\(.*\)\()\)/\2/'"
interval => 7
type => "server_status"
id => "server_status"
}
exec {
command => "mongo mongodb://127.0.0.1:27617/userhabit -u root -p hmsecroot1@ --authenticationDatabase admin --eval 'db.session.stats()' --quiet | sed 's/\(NumberLong([[:punct:]]\?\)\([[:digit:]]*\)\([[:punct:]]\?)\)/\2/' | sed 's/\(ISODate(\)\(.*\)\()\)/\2/' | sed 's/Timestamp(\([0-9]*\),[[:space:]]*[[:digit:]]*)/\1/' | sed 's/BinData([[:digit:]],\([^,]*\))/\1/'"
interval => 7
type => "coll_stats"
id => "coll_stats_session"
}
exec {
command => "mongo mongodb://127.0.0.1:27617/userhabit -u root -p hmsecroot1@ --authenticationDatabase admin --eval 'db.event.stats()' --quiet | sed 's/\(NumberLong([[:punct:]]\?\)\([[:digit:]]*\)\([[:punct:]]\?)\)/\2/' | sed 's/\(ISODate(\)\(.*\)\()\)/\2/' | sed 's/Timestamp(\([0-9]*\),[[:space:]]*[[:digit:]]*)/\1/' | sed 's/BinData([[:digit:]],\([^,]*\))/\1/'"
interval => 7
type => "coll_stats"
id => "coll_stats_event"
}
exec {
command => "mongo mongodb://127.0.0.1:27617/userhabit -u root -p hmsecroot1@ --authenticationDatabase admin --eval 'db.stats()' --quiet | sed 's/\(NumberLong([[:punct:]]\?\)\([[:digit:]]*\)\([[:punct:]]\?)\)/\2/' | sed 's/\(ISODate(\)\(.*\)\()\)/\2/' | sed 's/Timestamp(\([0-9]*\),[[:space:]]*[[:digit:]]*)/\1/' | sed 's/BinData([[:digit:]],\([^,]*\))/\1/'"
interval => 7
type => "db_stats"
id => "db_stats"
}
exec {
command => "mongotop -uri=mongodb://127.0.0.1:27617/userhabit -u root -p hmsecroot1@ --authenticationDatabase=admin --json -n 1"
interval => 7
type => "mongotop"
id => "mongotop"
}
exec {
command => "mongostat -uri=mongodb://127.0.0.1:27617/userhabit -u root -p hmsecroot1@ --authenticationDatabase=admin --json -n 1"
interval => 7
type => "mongostat"
id => "mongostat"
}
}
filter {
json {
source => "message"
}
}
output {
if [type] == "server_status" or [type] == "coll_stats" or [type] == "db_stats" or [type] == "mongotop" or [type] == "mongostat" {
elasticsearch {
hosts => ["10.227.108.249:9200"]
index => "test-%{type}-metricbeat-%{+YYYY.MM.dd}"
}
}
else {
elasticsearch {
hosts => ["10.227.108.249:9200"]
index => "test-else-metricbeat-%{+YYYY.MM.dd}"
}
}
}
5. 로그 확인 / data 확인
로그확인
tail -f /var/log/logstash/ligstash-plain.log
※ logstash 가 실행할때 시간이 걸려서 바로 로그를 보려하면 파일이 존재 안할 수 있다
data 확인 : 모니터링 서버에 있는 elasticsearch 에서 data 가 들어오는지 확인
# 시간순으로 가장 최근것
curl '10.227.108.249:9200/test-coll_stats-metricbeat*/_search?pretty' -H 'Content-Type: application/json' -d '
{
"sort" : [{"@timestamp" : {"order":"desc"}}], "query":{"term":{"host":"dmcesv01"}}
}
'