Reading Phoenix Upserted data directly from Hbase

[Vaghawan Ojha]

Hi,

I've few phoenix tables created from Phoenix itself, they do fine with the
Phoenix, however, when I try to scan the data from hbase shell, the binary
strings get printed instead of the real values, like the one I can see in
the phoenix.

Also, there are cases when I want to fetch them directly from hbase and
work with spark. I guess I'm doing something wrong with the configuration
of phoenix, or is this the expected result?

I'm using phoenix-4.12.0-HBase-1.2 .

Any reply would be appreciated.

Thanks

Re: Help: setting hbase row timestamp in phoenix upserts ?

[James Taylor]

The only way I can think of accomplishing this is by using the raw HBase
APIs to write the data but using our utilities to write it in a Phoenix
compatible manner. For example, you could run an UPSERT VALUES statement,
use the PhoenixRuntime.getUncommittedDataIterator()method to get the Cells
that would have been written, update the Cell timestamp as needed, and do
an htable.batch() call to commit them.

On Wed, Nov 29, 2017 at 11:46 AM Pedro Boado  wrote:

> Hi,
>
> I'm looking for a little bit of help trying to get some light over
> ROW_TIMESTAMP.
>
> Some background over the problem ( simplified ) : I'm working in a project
> that needs to create a "enriched" replica of a RBDMS table based on a
> stream of cdc changes off that table.
>
> Each cdc event contains the timestamp of the change plus all the column
> values 'before' and 'after' the change . And each event is pushed to a
> kafka topic.  Because of certain "non-negotiable" design decisions kafka
> guarantees delivering each event at least once, but doesn't guarantee
> ordering for changes over the same row in the source table.
>
> The final step of the kafka-based flow is sinking the information into
> HBase/Phoenix.
>
> As I cannot get in order delivery guarantee from Kafka I need to use the
> cdc event timestamp to ensure that HBase keeps the latest change over a row.
>
> This fits perfectly well with an HBase table design with VERSIONS=1 and
> using the source event timestamp as HBase row/cells timestamp
>
> The thing is that I cannot find a way to define the value of the HBase
> cell from a Phoenix upsert.
>
> I came across the ROW_TIMESTAMP functionality, but I've just found ( I'm
> devastated now ) that the ROW_TIMESTAMP columns store the date in both
> hbase's cell timestamp and in the primary key, meaning that I cannot
> leverage that functionality to keep only the latest change.
>
> Is there a way of defining hbase's row timestamp when doing the UPSERT -
> even by setting it through some obscure hidden jdbc property - ?
>
> I want to avoid by all means doing a checkAndPut as the volume of changes
> is going to be quite bug.
>
>
>
> --
> Un saludo.
> Pedro Boado.
>

Re: Re-distribute Phoenix 4.8 or later without any License issues due to nested libraries

[Josh Elser]

Please note that the following does not constitute legal advice. Please 
consult a lawyer.


--

The LICENSE and NOTICE files are provided, as a convenience, to users of 
Apache Phoenix so that they are aware of the licenses and copyright of 
the third-party software that is distributed as a part of Apache Phoenix.


If you choose to redistribute Apache Phoenix as a part of some other 
product, it is your responsibility to investigate all licenses (and 
their requirements imposed upon you) and ensure that you comply with all 
of them, both Apache Phoenix's and bundled third-party software.


Please be aware that there are also guidelines in how you refer to 
Apache Phoenix when it is modified/redistributed: 
https://www.apache.org/foundation/marks/faq/


On 11/29/17 2:24 AM, Mallieswari Dineshbabu wrote:

Hi Team,

We would like to include the following Phoenix *distribution* into our 
product and re-distribute the same.


https://archive.apache.org/dist/phoenix/apache-phoenix-4.12.0-HBase-1.2/bin/

Though the License is listed as Apache License Version 2.0, there are 
*many nested dependencies in the license file*.


https://github.com/apache/phoenix/blob/4.12-HBase-1.2/dev/release_files/LICENSE 



Could you please clarify us the following,

 1. Can we *re-distribute* Phoenix distribution after doing some minor
changes in conf files like sqlline.py to mention Hadoop path? (We
will ship the License file as well)
 2. Will the nested libraries mentioned in License agreement will *not
result with any problem* for us once we distribute?

Please let me know in case of any queries,


--
Thanks and regards
Mallieswari D

case of bulk loader use CurrentSCN

[Bulvik, Noam]

Hi
Is it possible to set CurrentSCN in a way that it will be used when inserting 
data using the bulk loader?

Assuming I have TTL of 3 month and I am loading historical data from a month 
ago using the bulk loader, is there a way to set the  CurrentSCN so that 
timestamp of the loaded data will be set to correct date and it will be deleted 
on the correct time
( I can't use raw timestamp because I have indexes on the table and it is 
mutable table so having row_timestamp is not allowed )

Regards,

Noam





PRIVILEGED AND CONFIDENTIAL
PLEASE NOTE: The information contained in this message is privileged and 
confidential, and is intended only for the use of the individual to whom it is 
addressed and others who have been specifically authorized to receive it. If 
you are not the intended recipient, you are hereby notified that any 
dissemination, distribution or copying of this communication is strictly 
prohibited. If you have received this communication in error, or if any 
problems occur with transmission, please contact sender. Thank you.

problem to run phoenix client 4.13.1 for CDH5.11.2 on windows

[Bulvik, Noam]

Hi
I am using JDBC UI client on windows. After I upgrade to latest parcel of 
phonix I got the following error (it did not happened on older parcels either 
when I compiled myself or when I used ones supplied by cloudera [4.7])

SEVERE: Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the 
Hadoop binaries.
at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:404)
at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:419)
at org.apache.hadoop.util.Shell.(Shell.java:412)
at org.apache.hadoop.util.StringUtils.(StringUtils.java:79)
at org.apache.hadoop.security.Groups.parseStaticMapping(Groups.java:168)
at org.apache.hadoop.security.Groups.(Groups.java:132)
at org.apache.hadoop.security.Groups.(Groups.java:100)
at 
org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:435)
at 
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:337)
at 
org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:304)
at 
org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:891)
at 
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:857)
   at 
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:724)
at 
org.apache.hadoop.hbase.security.User$SecureHadoopUser.(User.java:293)
at org.apache.hadoop.hbase.security.User.getCurrent(User.java:191)
at 
org.apache.phoenix.jdbc.PhoenixEmbeddedDriver$ConnectionInfo.(PhoenixEmbeddedDriver.java:504)
at 
org.apache.phoenix.jdbc.PhoenixEmbeddedDriver$ConnectionInfo.create(PhoenixEmbeddedDriver.java:312)
at 
org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:232)
at 
org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:150)
at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
at workbench.db.DbDriver.connect(DbDriver.java:513)
at workbench.db.ConnectionMgr.connect(ConnectionMgr.java:255)
at workbench.db.ConnectionMgr.getConnection(ConnectionMgr.java:182)
at 
workbench.gui.profiles.ConnectionGuiHelper$1.run(ConnectionGuiHelper.java:142)









Noam




PRIVILEGED AND CONFIDENTIAL
PLEASE NOTE: The information contained in this message is privileged and 
confidential, and is intended only for the use of the individual to whom it is 
addressed and others who have been specifically authorized to receive it. If 
you are not the intended recipient, you are hereby notified that any 
dissemination, distribution or copying of this communication is strictly 
prohibited. If you have received this communication in error, or if any 
problems occur with transmission, please contact sender. Thank you.