guluo created AMBARI-25880:
------------------------------
Summary: For hadoop_logs collection of Infra-solr, indexed
log_message field is too large to copy to key_log_message field
Key: AMBARI-25880
URL: https://issues.apache.org/jira/browse/AMBARI-25880
Project: Ambari
Issue Type: Bug
Components: ambari-infra, ambari-logsearch
Affects Versions: 2.7.3
Reporter: guluo
The hadoop_logs collection of Infra-solr is used to store logs generated by
services, such as hadoop、spark， etc.
However, Infra-solr will report an error when the length of the single-line log
generated by services exceeds 2^15-2, Because Infra-solr cannot copy the data
of field log_message to field key_log_messag
{code:java}
//代码占位符
org.apache.solr.common.SolrException: Exception writing document id
54d786cd-f6a7-4746-9b17-e9b2b0fe389a to the index; possible analysis error:
Document contains at least one immense term in field="action" (whose UTF8
encoding is longer than the max length 32766), all of which were skipped.
Please correct the analyzer to not produce such terms. The prefix of the first
immense term is: '[98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98,
98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98]...', original
message: bytes can be at most 32766 in length; got 32767. Perhaps the document
has an indexed string field (solr.StrField) which is too large {code}
*Reproduce Step:*
1 Create a json File named data.json
[
{
"log_message": "********",
"id": "54d786cd-f6a7-4746-9b17-e9b2b0fe3efa"
}
]
Note: the content of field log_message is a randomly generated string with a
length exceeding 2^15-2
2 Update the hadoop_logs
curl http://ip:port/solr/hadoop_logs/update?commit=true -H
'Content-type:application/json' --data-binary @data.json
3 Get the error message mentioned above
*similar issue：*
AMBARI-24067: Log Search: indexed string field could be too large for audit_logs
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
