[
https://issues.apache.org/jira/browse/HIVE-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16473397#comment-16473397
]
Hive QA commented on HIVE-19490:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923088/HIVE-19490.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10889/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10889/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10889/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Tests exited with: ExecutionException: java.util.concurrent.ExecutionException:
org.apache.hive.ptest.execution.ssh.SSHExecutionException: RSyncResult
[localFile=/data/hiveptest/logs/PreCommit-HIVE-Build-10889/succeeded/221_UTBatch_service_8_tests,
remoteFile=/home/hiveptest/35.225.119.98-hiveptest-2/logs/, getExitCode()=255,
getException()=null, getUser()=hiveptest, getHost()=35.225.119.98,
getInstance()=2]: 'Warning: Permanently added '35.225.119.98' (ECDSA) to the
list of known hosts.
receiving incremental file list
./
TEST-221_UTBatch_service_8_tests-TEST-org.apache.hive.service.auth.TestLdapAtnProviderWithMiniDS.xml
0 0% 0.00kB/s 0:00:00
93,274 100% 1.31MB/s 0:00:00 (xfr#1, to-chk=12/14)
TEST-221_UTBatch_service_8_tests-TEST-org.apache.hive.service.auth.ldap.TestCustomQueryFilter.xml
0 0% 0.00kB/s 0:00:00
89,239 100% 1.23MB/s 0:00:00 (xfr#2, to-chk=11/14)
TEST-221_UTBatch_service_8_tests-TEST-org.apache.hive.service.auth.ldap.TestQuery.xml
0 0% 0.00kB/s 0:00:00
89,224 100% 854.24kB/s 0:00:00 (xfr#3, to-chk=10/14)
TEST-221_UTBatch_service_8_tests-TEST-org.apache.hive.service.auth.ldap.TestUserFilter.xml
0 0% 0.00kB/s 0:00:00
89,213 100% 845.85kB/s 0:00:00 (xfr#4, to-chk=9/14)
TEST-221_UTBatch_service_8_tests-TEST-org.apache.hive.service.auth.ldap.TestUserSearchFilter.xml
0 0% 0.00kB/s 0:00:00
89,650 100% 634.41kB/s 0:00:00 (xfr#5, to-chk=8/14)
TEST-221_UTBatch_service_8_tests-TEST-org.apache.hive.service.cli.TestCLIServiceConnectionLimits.xml
0 0% 0.00kB/s 0:00:00
90,521 100% 635.97kB/s 0:00:00 (xfr#6, to-chk=7/14)
TEST-221_UTBatch_service_8_tests-TEST-org.apache.hive.service.cli.TestCLIServiceRestore.xml
0 0% 0.00kB/s 0:00:00
88,990 100% 625.21kB/s 0:00:00 (xfr#7, to-chk=6/14)
TEST-221_UTBatch_service_8_tests-TEST-org.apache.hive.service.cli.TestHiveSQLException.xml
0 0% 0.00kB/s 0:00:00
89,735 100% 625.94kB/s 0:00:00 (xfr#8, to-chk=5/14)
maven-test.txt
0 0% 0.00kB/s 0:00:00
5,969 100% 41.64kB/s 0:00:00 (xfr#9, to-chk=4/14)
logs/
logs/derby.log
0 0% 0.00kB/s 0:00:00
985 100% 6.87kB/s 0:00:00 (xfr#10, to-chk=1/14)
logs/hive.log
0 0% 0.00kB/s 0:00:00
42,434,560 1% 40.47MB/s 0:01:41
99,123,200 2% 47.24MB/s 0:01:25
154,894,336 3% 49.24MB/s 0:01:20
210,042,880 4% 49.99MB/s 0:01:18
266,403,840 6% 53.31MB/s 0:01:12
322,830,336 7% 53.28MB/s 0:01:11
373,555,200 8% 52.08MB/s 0:01:12
417,169,408 9% 49.41MB/s 0:01:15
464,912,384 10% 47.33MB/s 0:01:17
515,833,856 12% 46.02MB/s 0:01:18
568,655,872 13% 46.52MB/s 0:01:16
622,854,144 14% 49.04MB/s 0:01:11
678,068,224 16% 50.78MB/s 0:01:08
734,887,936 17% 52.20MB/s 0:01:05
791,707,648 18% 53.17MB/s 0:01:03
848,527,360 20% 53.75MB/s 0:01:01
905,969,664 21% 54.34MB/s 0:00:59
963,510,272 22% 54.48MB/s 0:00:58
1,021,083,648 24% 54.66MB/s 0:00:57
1,071,841,280 25% 53.24MB/s 0:00:57
1,120,927,744 26% 51.25MB/s 0:00:59
1,177,026,560 27% 49.17MB/s 0:01:00
1,234,239,488 29% 49.09MB/s 0:00:59
1,275,068,416 30% 45.34MB/s 0:01:03
1,326,972,928 31% 45.64MB/s 0:01:02
1,385,332,736 32% 47.73MB/s 0:00:58
1,419,509,760 33% 41.53MB/s 0:01:06
1,448,083,456 34% 39.26MB/s 0:01:09
1,486,880,768 35% 35.98MB/s 0:01:14
1,532,231,680 36% 33.05MB/s 0:01:19
1,579,188,224 37% 36.48MB/s 0:01:10
1,627,947,008 38% 41.95MB/s 0:01:00
1,674,313,728 39% 43.24MB/s 0:00:57
1,729,953,792 40% 45.60MB/s 0:00:53
1,786,970,112 42% 48.24MB/s 0:00:49
1,844,674,560 43% 50.31MB/s 0:00:46
1,902,313,472 44% 54.37MB/s 0:00:41
1,958,641,664 46% 54.52MB/s 0:00:40
1,995,964,416 47% 48.66MB/s 0:00:44
2,041,839,616 48% 45.30MB/s 0:00:47
2,086,830,080 49% 42.38MB/s 0:00:49
2,136,309,760 50% 40.83MB/s 0:00:50
2,160,852,992 51% 38.62MB/s 0:00:52
2,211,446,784 52% 40.28MB/s 0:00:48
2,267,807,744 53% 42.99MB/s 0:00:44
2,325,774,336 54% 44.99MB/s 0:00:41
2,383,675,392 56% 53.15MB/s 0:00:33
2,442,002,432 57% 55.00MB/s 0:00:31
2,498,166,784 59% 54.95MB/s 0:00:30
2,546,663,424 60% 52.69MB/s 0:00:31
2,573,729,792 60% 44.32MB/s 0:00:36
2,595,815,424 61% 35.73MB/s 0:00:44
2,646,409,216 62% 34.44MB/s 0:00:44
2,703,491,072 63% 36.43MB/s 0:00:40
2,761,162,752 65% 44.56MB/s 0:00:32
2,818,572,288 66% 53.15MB/s 0:00:25
2,876,637,184 67% 54.93MB/s 0:00:24
2,934,734,848 69% 55.17MB/s 0:00:22
2,992,832,512 70% 55.28MB/s 0:00:21
3,050,897,408 72% 55.42MB/s 0:00:20
3,108,798,464 73% 55.38MB/s 0:00:19
3,166,437,376 74% 55.27MB/s 0:00:18
3,222,994,944 76% 54.86MB/s 0:00:17
3,281,354,752 77% 54.93MB/s 0:00:16
3,339,583,488 78% 55.01MB/s 0:00:15
3,392,503,808 80% 53.76MB/s 0:00:15
3,438,051,328 81% 51.20MB/s 0:00:15
3,487,793,152 82% 49.13MB/s 0:00:14
3,540,910,080 83% 47.92MB/s 0:00:14
3,596,484,608 84% 48.62MB/s 0:00:12
3,652,255,744 86% 51.06MB/s 0:00:11
3,708,715,008 87% 52.67MB/s 0:00:09
3,765,239,808 88% 53.48MB/s 0:00:08
3,821,797,376 90% 53.75MB/s 0:00:07
3,877,175,296 91% 53.56MB/s 0:00:06
3,931,668,480 92% 53.05MB/s 0:00:05
3,989,372,928 94% 53.36MB/s 0:00:04
4,039,868,416 95% 51.91MB/s 0:00:03
4,091,510,784 96% 51.11MB/s 0:00:02
4,141,350,912 97% 50.02MB/s 0:00:01 Timeout, server 35.225.119.98 not
responding.
rsync: connection unexpectedly closed (4174116544 bytes received so far)
[receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(226)
[receiver=3.1.1]
rsync: connection unexpectedly closed (845 bytes received so far) [generator]
rsync error: unexplained error (code 255) at io.c(226) [generator=3.1.1]
ssh: connect to host 35.225.119.98 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
ssh: connect to host 35.225.119.98 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
ssh: connect to host 35.225.119.98 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
ssh: connect to host 35.225.119.98 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12923088 - PreCommit-HIVE-Build
> Locking on Insert into for non native and managed tables.
> ---------------------------------------------------------
>
> Key: HIVE-19490
> URL: https://issues.apache.org/jira/browse/HIVE-19490
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
> Reporter: slim bouguerra
> Assignee: slim bouguerra
> Priority: Major
> Labels: druid, locking
> Attachments: HIVE-19490.2.patch, HIVE-19490.patch
>
>
> Current state of the art:
> Managed non native table like Druid Tables, will need to get a Lock on Insert
> into or insert Over write. The nature of this lock is set to Exclusive by
> default for any non native table.
> This implies that Inserts into Druid table will Lock any read query as well
> during the execution of the insert into. IMO this lock (on insert into) is
> not needed since the insert statement is appending data and the state of
> loading it is managed partially by Hive Storage handler hook and part of it
> by Druid.
> What i am proposing is to relax the lock level to shared for all non native
> tables on insert into operations and keep it as Exclusive Write for insert
> Overwrite for now.
>
> Any feedback is welcome.
> cc [~ekoifman] / [~ashutoshc] / [~jdere] / [~hagleitn]
> Also am not sure what is the best way to unit test this currently am using
> debugger to check if locks are what i except, please let me know if there is
> a better way to do this.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
