Supritha created CARBONDATA-2875:
------------------------------------
Summary: Two different threads are overwriting the same
carbondatafile
Key: CARBONDATA-2875
URL: https://issues.apache.org/jira/browse/CARBONDATA-2875
Project: CarbonData
Issue Type: Bug
Components: data-load
Affects Versions: 1.5.0
Environment: Spark 2.2
Reporter: Supritha
Steps :
sparS.sql("create external table if not exists thread_x stored by 'carbondata'
location
'D:/KernelTest_New/KernetTest/KernelTest_New/KernetTest/thread/thread'").show();
sparS.sql("select count(*) from thread_x").show(); ----> 300000 record count
try{
System.out.println("count called after write");
Thread t1 = new Thread(new Trial(sparS));
Thread t2 = new Thread(new Trial(sparS));
//Each thread is holding 2 records each
t1.setName("T1");
t2.setName("T2");
t1.start();
Thread.sleep(10);
t2.start();
t1.join();
t2.join();
sparS.sql("select count(*) from thread_x").show();
System.out.println("counrt called after load");
Actual Issue : sparS.sql("select count(*) from thread_x").show(); -----> 300002
Which implies Two different threads are overwriting the same carbondatafile.
Expected : 300004 record count should be displayed.
ie A carbonfile has to be created for each thread.
>From logs its is seen that two different threads are overwriting the same
>carbondatafile
Line 298: 18/08/22 16:58:10 INFO CarbonUtil: WriterForwardPool: thread_x
Copying
C:\Users\SWX593~1\AppData\Local\Temp\\carbon16096431985350_0\Fact\Part0\Segment_1534937289563\0\part-0-0_batchno0-0-1534937289563-1534937289561.carbondata
--> D:/KernelTest_New/KernetTest/KernelTest_New/KernetTest/thread/thread
Line 302: 18/08/22 16:58:10 INFO CarbonUtil: WriterForwardPool: thread_x
Copying
C:\Users\SWX593~1\AppData\Local\Temp\\carbon16096431985350_0\Fact\Part0\Segment_1534937289563\0\0_batchno0-0-1534937289563-1534937289561.carbonindex
--> D:/KernelTest_New/KernetTest/KernelTest_New/KernetTest/thread/thread
Line 374: 18/08/22 16:58:11 INFO CarbonUtil: WriterForwardPool: thread_x
Copying
C:\Users\SWX593~1\AppData\Local\Temp\\carbon16096980581610_0\Fact\Part0\Segment_1534937289563\0\part-0-0_batchno0-0-1534937289563-1534937289561.carbondata
--> D:/KernelTest_New/KernetTest/KernelTest_New/KernetTest/thread/thread
Line 378: 18/08/22 16:58:11 INFO CarbonUtil: WriterForwardPool: thread_x
Copying
C:\Users\SWX593~1\AppData\Local\Temp\\carbon16096980581610_0\Fact\Part0\Segment_1534937289563\0\0_batchno0-0-1534937289563-1534937289561.carbonindex
--> D:/KernelTest_New/KernetTest/KernelTest_New/KernetTest/thread/thread
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
