This is an automated email from the ASF dual-hosted git repository. jenniferdai pushed a commit to branch orcdoc in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit e59ea576a04469b19603c71bdba058f61491b56b Author: Jennifer Dai <[email protected]> AuthorDate: Mon Apr 1 11:54:28 2019 -0700 Add ORC docs --- docs/extensions.rst | 3 ++- docs/pluggable_record_reader.rst | 46 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+), 1 deletion(-) diff --git a/docs/extensions.rst b/docs/extensions.rst index 43872b8..b9c42bd 100644 --- a/docs/extensions.rst +++ b/docs/extensions.rst @@ -26,4 +26,5 @@ This section provides an overview of options to extend Pinot code to make Pinot pluggable_streams segment_fetcher - pluggable_storage \ No newline at end of file + pluggable_storage + pluggable_record_reader \ No newline at end of file diff --git a/docs/pluggable_record_reader.rst b/docs/pluggable_record_reader.rst new file mode 100644 index 0000000..51c2fa5 --- /dev/null +++ b/docs/pluggable_record_reader.rst @@ -0,0 +1,46 @@ +.. +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. +.. + +.. _pluggable-record-reader: + +Pluggable Record Reader +======================= + +Pinot enables its users to write and use alternate record readers to generate segments. + +ORC is supported. + +* `ORC <https://orc.apache.org/>`_ + + +New Storage Type implementation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +In order to add a new type of record reader, implement the following class: + +X extends `RecordReader <https://github.com/apache/incubator-pinot/blob/9912b47959c1a9c0915588b4ac60f8ca836a8e27/pinot-core/src/main/java/org/apache/pinot/core/data/readers/RecordReader.java>`_ + +Configurations +^^^^^^^^^^^^^^ +The following property is to be set during segment generation in your Hadoop properties. + +record.reader.path: ${FULL_PATH_OF_YOUR_RECORD_READER_CLASS} + +For example, for ORC, it would be: + +record.reader.path: org.apache.pinot.orc.data.readers.ORCRecordReader \ No newline at end of file --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
