On a recent, fresh installation of LogInsight I observed that no data from a set of hosts was coming into the LogInsight server.
Debugging showed not issues related to the usual suspects of DNS, NTP, firewall or network which left me a bit puzzled.
I found the answer by going through the logs of the loginsight agent (usually in /var/log/loginsight-agent/liagent_<date>):
2021-07-20 07:08:22.319018 0x00007ff1c1a74700 <trace> DataController:167 | Creating cfapi transport
2021-07-20 07:08:22.319039 0x00007ff1c1a74700 <trace> Config:292 | Read config param [server].hostname = logs.lab.why-did-it.fail
2021-07-20 07:08:22.319066 0x00007ff1c1a74700 <trace> Config:339 | Read config param [server].ssl = no
2021-07-20 07:08:22.319114 0x00007ff1c1a74700 <trace> Config:224 | Read config param [server].port = 9000
2021-07-20 07:08:22.319158 0x00007ff1c1a74700 <trace> Config:257 | Configuration key [server].reconnect is not specified. Using default: 30
2021-07-20 07:08:22.319217 0x00007ff1c1a74700 <trace> Config:351 | Configuration key [server].compress is not specified. Using default: yes
2021-07-20 07:08:22.319238 0x00007ff1c1a74700 <trace> Config:351 | Configuration key [server].central_config is not specified. Using default: yes
2021-07-20 07:08:22.325292 0x00007ff1c1a74700 <trace> DataController:105 | Starting transport...
2021-07-20 07:08:22.325944 0x00007ff1b3fff700 <trace> Logger:209 | Thread "CFApiTransport" has id 0x7ff1b3fff700
2021-07-20 07:08:22.325999 0x00007ff1b3fff700 <trace> CFApiTransport:130 | Connecting to server logs.lab.why-did-it.fail:9000
2021-07-20 07:08:22.326021 0x00007ff1c1a74700 <trace> AgentDaemon:437 | AgentDaemon configured successfully
2021-07-20 07:08:22.326055 0x00007ff1c1a74700 <trace> AgentDaemon:378 | AgentDaemon started successfully
2021-07-20 07:08:22.343945 0x00007ff1b3fff700 <trace> CFApiTransport:152 | Connection to logs.lab.why-did-it.fail:9000 successfully established
2021-07-20 07:08:22.349596 0x00007ff1b3fff700 <warng> CurlConnection:181 | Status response code from logs.lab.why-did-it.fail : 403 Only SSL connections are allowed
There is a new default behavior in greenfield installation of recent LogInsight installations that I wasn’t aware of:
Newer installations Enforce SSL-Only Connections, this prevents LogInsight from accepting data coming unsecured via port 9000 (the default for CFAPI without SSL). The setting is described the documentation.
Once you toggle the setting the data will be accepted (no reboot required).
Comments