I think it is just either issue of memory allocation for transactional data
creation or it could be related to parsing the huge JSON.
Can you try to run the apoc.load.json and just return count(*) without
creating data to see how long that takes?
And then you can combine apoc.periodic.iterate with apoc.load.json to batch
your data creation
call apoc.periodic.iterate("
CALL apoc.load.json('file:///root/output/1532360242087/devices1.json')
YIELD value AS row UNWIND row.devices as val RETURN val
","
MERGE(dev:device{dwwn:val.device_wwn})
ON CREATE SET dev.title='device',dev.sid=val.sid,dev.scsi_3_persistent_res
erve=val.scsi_3_persistent_reserve,
...
", {batchSize:10000, iterateList:true, parallel:false});
you can leave off the "RETURN count(dev)"
and you can also use SET dev += val
to set all properties of the "val" map at once
On Wed, Jul 25, 2018 at 5:27 PM, Elyas Mohamed <[email protected]>
wrote:
> Hi,
>
> We have got about 80000 roes to be loaded . We have created unique
> constrain for anode and try to load 80000 rows it takes more than 30
> minutes. Any help appreciated. Below the query
>
> CREATE CONSTRAINT ON (n:device) ASSERT n.dwwn IS UNIQUE
>
> CREATE INDEX ON : device (dwwn);
>
>
>
> CALL apoc.load.json("file:///root/output/1532360242087/devices1.json")
> YIELD value AS row UNWIND row.devices as val MERGE(dev:device{dwwn:val.devi
> ce_wwn})
>
>
> ON CREATE SET dev.title='device',dev.sid=val
> .sid,dev.scsi_3_persistent_reserve=val.scsi_3_persistent_res
> erve,dev.raid_group_information_number_of_failing_members=va
> l.raid_group_information_number_of_failing_
>
>
> members,dev.megabytes=val.megabytes,dev.sqar_mode=val.sqar_m
> ode,dev.mirror_configuration_information_mirror_type=val.mir
> ror_configuration_information_mirror_type,dev.device_
> emulation_type=val.device_emulation_type,dev.product_
> revision=val.product_revision,dev.star_mode=val.star_mode,
> dev.device_sub_system_id=val.device_sub_system_id,dev.
> mirror_set_type=val.mirror_set_type,dev.mirror_set_da_
> status=val.mirror_set_da_status,dev.device_user_pinned=
> val.device_user_pinned,dev.product_id=val.product_id,dev.
> raid_group_information_raid_type=val.raid_group_information_raid_type,dev.
> raid_group_information_disk_group_number=val.raid_group_
> information_disk_group_number,dev.device_physical_name=val.
> device_physical_name,dev.device_configuration=val.
> device_configuration,dev.device_defined_label=val.
> device_defined_label,dev.device_serial_id=val.device_
> serial_id,dev.host_cache_registered=val.host_cache_
> registered,dev.device_status=val.device_status,dev.dif1_
> flag=val.dif1_flag,dev.device_external_identity_device_wwn=
> val.device_external_identity_device_wwn,dev.`512_byte_
> blocks`=val.`512_byte_blocks`,dev.extent_based_clone=val.
> extent_based_clone,dev.raid_group_information_mirror_
> number=val.raid_group_information_mirror_number,dev.
> encapsulated_device_flags=val.encapsulated_device_flags,dev.
> device_block_size=val.device_block_size,dev.cylinders=val.
> cylinders,dev.kilobytes=val.kilobytes,dev.device_service_
> state=val.device_service_state,dev.vendor_id=val.
> vendor_id,dev.as400_gk=val.as400_gk,dev.gatekeeper_
> device=val.gatekeeper_device,dev.star_recovery_state=val.
> star_recovery_state,dev.device_capacity_tracks=val.
> device_capacity_tracks,dev.raid_group_information_raid_
> group_service_state=val.raid_group_information_raid_group_
> service_state,dev.device_symmetrix_name=val.device_
> symmetrix_name,dev.device_capacity_megabytes=val.device_
> capacity_megabytes,dev.host_access_mode=val.host_access_
> mode,dev.raid_group_information_disk_group_name=
> val.raid_group_information_disk_group_name,dev.optimized_
> read_miss=val.optimized_read_miss,dev.raid_group_
> information_device_position=val.raid_group_information_
> device_position,dev.encapsulated_wwn=val.encapsula
> ted_wwn,dev.raid_group_information_protection_level=val.
> raid_group_information_protection_level,dev.device_capacity_
> kilobytes=val.device_capacity_kilobytes,dev.number_of_raid_
> groups=val.number_of_raid_groups,dev.encapsulated_device
> _name=val.encapsulated_device_name,dev.device_capacity_
> cylinders=val.device_capacity_cylinders,dev.attached_vdev_
> tgt_device=val.attached_vdev_tgt_device,dev.device_sa_
> status=val.device_sa_status,dev.device_defined_label_type=
> val.device_defined_label_type,dev.dynamic_rdf_capability=
> val.dynamic_rdf_capability,dev.device_capacity_geometry_l
> imited=val.device_capacity_geometry_limited,dev.mirror_confi
> guration_information_mirror_status=val.mirror_configuratio
> n_information_mirror_status,dev.dynamic_spare_invoked=val.dy
> namic_spare_invoked,dev.trackscylinder=val.trackscylinder,de
> v.star_recovery_capability=val.star_recovery_capability,dev.
> device_tags=val.device_tags,dev.mirror_configuration_infor
> mation_mirror_number=val.mirror_configuration_information_
> mirror_number,dev.encapsulated_array_id=val.encapsulated_arr
> ay_id,dev.sectorstrack=val.sectorstrack,dev.encapsulated_dev
> ice=val.encapsulated_device,dev.attached_bcv_device=val.
> attached_bcv_device,dev.device_capacity_512_byte_
> blocks=val.device_capacity_512_byte_blocks,dev.raid_
> group_information_engine_number=val.raid_group_information_
> engine_number,dev.cache_partition_name=val.cache_partition_
> name,dev.mirror_set_inv._tracks=val.mirror_set_inv,dev.geometry=val.geometry
> return count(dev);
>
> --
> You received this message because you are subscribed to the Google Groups
> "Neo4j" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
