본문 바로가기

내가 당면한 문제와 해결방안

 (81)
logstash 설정 Logstash 서버 211.233.74.227 pg table : expedia_region_union → ES index : region_union 연동 conf vi /etc/logstash/conf.d/pg-logstash-es-05-18-mon.conf jvm.options vi /etc/logstash/jvm.options : -Xms12g -Xmx12g로 변경 startup.options vi /etc/logstash/startup.options : 설정 확인 pipelines.yml 디폴트 파일 /etc/logstash/pipelines.yml을 /etc/logstash/pipelines-default.yml로 변경 (해당 파일은 conf.d 경로 아래있는 conf 파일 모두 읽음) /et..
logstash conf input { jdbc { jdbc_connection_string => "jdbc:postgresql://IP:25432/atlasdb" jdbc_user => "USER" jdbc_password => "PW" jdbc_driver_library => "/lib/postgresql-42.2.12.jar" jdbc_validate_connection => true jdbc_driver_class => "org.postgresql.Driver" columns_charset => {"region_name_kr" => "UTF-8"} schedule => "*/10 * * * *" statement => "SELECT region_id, st_asgeojson(center_geo_point)::text as..
alias 변경 예시 curl -XPOST 'https://atlas-search.회사.com/_aliases?pretty' -H 'Content-Type: application/json' -d' { "actions" : [ { "remove" : { "index" : "region_union_20200513_1", "alias" : "region_union" } }, { "add" : { "index" : "expedia_region_union_backup_047", "alias" : "region_union" } } ] }'
logstash nested value to root value require 'json' begin jsonn_json = JSON.parse(event.get('jsonn').to_s || '{}') jsonn_json.each {|k,v| event.set(k, v) } event.remove('jsonn') rescue Exception => e event.tag('something happens in jsonn') end : discuss.elastic.co/t/how-to-dynamically-move-nested-key-value-to-root-level/180006/2?u=ohoroyoi
ruby text to json location = {lat: event.get('center_latitude').to_f, lon: event.get('center_longitude').to_f} event.set('location', JSON(location.to_json)) logstash ruby filter
create index with mapping text, keyword 는 따로 선 매핑 필요없으나, geo_shape, geo_point같은 경우 필요 { "mappings": { "doc": { "properties": { "city_ancestors":{ "type":"nested", "properties":{ "code":{ "type":"text", "fields":{ "keyword":{ "type":"keyword", "ignore_above":256 } } }, "id":{ "type":"text", "fields":{ "keyword":{ "type":"keyword", "ignore_above":256 } } }, "name":{ "type":"text", "fields":{ "keyword":{ "type":"keyword", "..
create geo_shape using latitude and longitude select st_makepoint(-46.635345, -23.61674)::json;
create index with mapping PUT 호스트명/인덱스명 { "mappings": { "_doc": { "properties": { "boundaries": { "type": "geo_shape", "ignore_malformed": true, "precision": "1.0m", "strategy": "recursive", "distance_error_pct": 0.009 }, "boundaries_nearby": { "type": "geo_shape", "ignore_malformed": true, "precision": "1.0m", "strategy": "recursive", "distance_error_pct": 0.009 }, "boundaries_nearby_auto": { "type": "geo_shape", "ignor..

티스토리툴바