Future versions of ElasticSearch will require Java 11.

The current ES version works just fine with Java 8:

novell   22175  4.7 12.8 4543824 913768 ?      Sl   Oct05 2125:36 /opt/novell/sentinel/jdk/jre/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,JRE -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.io.tmpdir=/tmp/elasticsearch-4956161078137255192 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:logs/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=32 -XX:GCLogFileSize=64m -Xms512m -Xmx512m -XX:MaxDirectMemorySize=268435456 -Des.path.home=/opt/novell/sentinel/3rdparty/elasticsearch -Des.path.conf=/opt/novell/sentinel/3rdparty/elasticsearch/config -Des.distribution.flavor=default -Des.distribution.type=tar -Des.bundled_jdk=true -cp /opt/novell/sentinel/3rdparty/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -d -Epath.data=/var/opt/novell/sentinel/3rdparty/elasticsearch/data -Epath.logs=/var/opt/novell/sentinel/log
novell   22233  0.0  0.0  45972  1732 ?        Sl   Oct05   0:00  \_ /opt/novell/sentinel/3rdparty/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller

Future versions of ElasticSearch will require Java 11.

The current ES version works just fine with Java 8:

novell   22175  4.7 12.8 4543824 913768 ?      Sl   Oct05 2125:36 /opt/novell/sentinel/jdk/jre/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,JRE -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.io.tmpdir=/tmp/elasticsearch-4956161078137255192 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:logs/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=32 -XX:GCLogFileSize=64m -Xms512m -Xmx512m -XX:MaxDirectMemorySize=268435456 -Des.path.home=/opt/novell/sentinel/3rdparty/elasticsearch -Des.path.conf=/opt/novell/sentinel/3rdparty/elasticsearch/config -Des.distribution.flavor=default -Des.distribution.type=tar -Des.bundled_jdk=true -cp /opt/novell/sentinel/3rdparty/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -d -Epath.data=/var/opt/novell/sentinel/3rdparty/elasticsearch/data -Epath.logs=/var/opt/novell/sentinel/log
novell   22233  0.0  0.0  45972  1732 ?        Sl   Oct05   0:00  \_ /opt/novell/sentinel/3rdparty/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller

Hi Wencheng, in addition to what Norbert provided...


I thought you would raise this with technical support.


Anyway, first of all my apologies:
www.microfocus.com/.../s85system-requirements.html

Data indexing

    Elasticsearch 7.7.0

    Download URL: www.elastic.co/.../elasticsearch-7-7-0

...this is wrong. Once you're on Sentinel 8.5, the ES rpm is upgraded to 7.10.2. So there was no reason to downgrade that to 7.7.0.
We actually corrected this link, so not sure why it reverted back to 7.7.0. I'm checking it internally, and get it changed.

And this will be updated as well:
www.microfocus.com/.../t45h61q5z43n.html

This is correct:
Elasticsearch 7.10.2

Download URL: www.elastic.co/.../elasticsearch-7-10-2


Once you upgrade your Sentinel appliance to 8.5, it will, should use ES rpm -> "version":"7.10.2".

I executed this, on my Sentinel 8.5 install:
novell@xxx:/opt/novell/sentinel/bin> ./elasticsearchRestClient.sh x.x.x.x 9200 GET /_nodes/process
Status Code: 200
{"_nodes":{"total":1,"successful":1,"failed":0},"cluster_name":"elasticsearch","nodes":{"_9-1lhWzRPafyVUkGf6Xcg":{"name":"xxx","transport_address":"x.x.x.x:9300","host":"x.x.x.x","ip":"x.x.x.x","version":"7.10.2","build_flavor":"default","build_type":"tar","build_hash":"747e1cc71def077253878a59143c1f785afa92b9","roles":["data","data_cold","data_content","data_hot","data_warm","ingest","master","ml","remote_cluster_client","transform"],"attributes":{"ml.machine_memory":"21039345664","xpack.installed":"true","transform.node":"true","ml.max_open_jobs":"20"},"process":{"refresh_interval_in_millis":1000,"id":3554,"mlockall":false}}}}

...it seems ES is now -> "version":"7.10.2".

I only have the single machine in my ES cluster, so one sentinel core server.

x.x.x.x -> ip address sentinel server (run this on core server, and external node, but using the associated ip address of that machine)


Sentinel 8.4 is still using 7.7.0:
novell@xxx:/opt/novell/sentinel/bin> ./elasticsearchRestClient.sh 127.0.0.1 9200 GET /_nodes/process
Status Code: 200
{"_nodes":{"total":1,"successful":1,"failed":0},"cluster_name":"elasticsearch","nodes":{"0E8sZ3whSNO8CTDLxn6USA":{"name":"xxx","transport_address":"127.0.0.1:9300","host":"127.0.0.1","ip":"127.0.0.1","version":"7.7.0","build_flavor":"default","build_type":"tar","build_hash":"81a1e9eda8e6183f5237786246f6dced26a10eaf","roles":["master","ingest","transform","data","remote_cluster_client","ml"],"attributes":{"ml.machine_memory":"16810741760","xpack.installed":"true","transform.node":"true","ml.max_open_jobs":"20"},"process":{"refresh_interval_in_millis":1000,"id":3598,"mlockall":false}}}}


One of our customers was getting this kibana error, since upgrade:
{"type":"log","@timestamp":"2021-09-26T15:17:42Z","tags":["warning","plugins","licensing"],"pid":20914,"message":"License information could not be obtained from Elasticsearch due to [illegal_argument_exception] request [/_xpack] contains unrecognized parameter: [accept_enterprise] :: {\"path\":\"/_xpack?accept_enterprise=true\",\"statusCode\":400,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"illegal_argument_exception\\\",\\\"reason\\\":\\\"request [/_xpack] contains unrecognized parameter: [accept_enterprise]\\\"}],\\\"type\\\":\\\"illegal_argument_exception\\\",\\\"reason\\\":\\\"request [/_xpack] contains unrecognized parameter: [accept_enterprise]\\\"},\\\"status\\\":400}\"} error"}

...and based on this url:
discuss.elastic.co/.../2

Their external es nodes were at SLES 12SP5 and elasticsearch at elasticsearch-7.7.0-1, so they didn't match. The core server was running 7.10.2. The ES version has to match on all nodes. OS can differ between nodes, however.
But make sure you check this as well, especially when adding external nodes to the ES cluster:
www.elastic.co/.../matrix

For example SLES 15 is not supported for ES 7.7, only from 7.9 onward.


We recommend strongly to have additional external nodes in the ES cluster, so it doesn't impact the Sentinel core server, check this:
www.microfocus.com/.../s85system-requirements.html

-> Elasticsearch Cluster Nodes section. It shows EPS level, and how many nodes you need.

And this:
www.microfocus.com/.../t45h61q5wyxv.html

www.microfocus.com/.../b1ndo3vk.html

www.microfocus.com/.../t462ks69vwjj.html

And this as well, when creating cluster:
www.microfocus.com/.../t4e5jeeqolch.html

-> Elasticsearch in Cluster Mode


Fix, when making sure all nodes were running same ES rpm, and OS was supported:
------------------------
Yes, that was the reason, kibana error, ES version mismatch ES nodes.

Just upgraded external ES nodes by:
- wget artifacts.elastic.co/.../elasticsearch-7.10.2-x86_64.rpm
- systemctl stop elasticsearch.service
- rpm -Uvh elasticsearch-7.10.2-x86_64.rpm
- systemctl daemon-reload
- systemctl start elasticsearch.service

...and kibana started right away.

The ES version is so critical.
------------------------


I hope this helps.


Thanks,


Henk