[orientdb] Re: Sticky sharding / sharding by sub-graph?
oh. I see there is a limit *You can create up to 32,767 (2^15-1) clusters in a database*... http://www.orientechnologies.com/docs/last/Tutorial-Clusters.html so I wouldn't be able to create shards/partitions for 10mln users :( as I understand the only hope for me is to rely on hazelcast to rebalance data closer to that particular app server node which serves particular user... not sure how efficient is it. -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[orientdb] vertex as document usage
hi, sorry for the noob question but i am using the graph database but some vertices i would like them to be documents. for example the user vertex has a lot of sub documents for the location, phones etc. can I do in my code ODocument od=vertex? and then how to save it? can I use the variables then like od.phones[0] or something? do I have to open a new transactional connection to a document database or work with my existing connection for graph? thanks -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[orientdb] Re: Sticky sharding / sharding by sub-graph?
well... I guess, I could use some generic class for all entities, so there is only one class - *Item*, with type and userId attributes on each node, ex: - *User*(id:123,name:john,age:99)-*Item* (type:user, userId:123, name:john,age:99) - *Purchase*(id:,userId:123,price:100)-*Item* (type:purchase, purchaseId:, userId:123, price:100) - *Account*(id:1,userId:123,balance:1)-*Item* (type:account, accountId:1, userId:123,balance:1) - etc. in that case, I hope, it would be possible to implement custom sharding strategy - based on attribute userId (all Items related to userId will be placed in the same partition - user, purchases of that user, accounts of that user, etc - that is what I'm looking for). But following this approach I lose object-oriented features of OrientDb API and custom types definition via schema, maybe I will loose something else too, which I don't know about yet :( maybe there is a better way to do such thing? -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[orientdb] Re: LET clause stocking only last record. Bug or am i doing something wrong ?
Noone can help me with this ? vineri, 20 martie 2015, 13:19:19 UTC+2, Mihai Pricop a scris: Hi, I am trying to use the LET clause to stock some information in an sub-query, keep the information relative to the record. After another select from the query where I save the variable, the variable holds the value only of the last record. For example, a query over the GratefulDeadConcerts: select *,$u from (select from (select expand(both()) from #9:8 limit 10) let $u = $current.performances) This results in: @rid@version@classperformancessong_typenametypeout_followed_by out_written_byout_sung_byin_followed_by$u#9:1 http://localhost:2480/studio/index.html#/database/GratefulDeadConcerts/browse/edit/9:1 25V5coverHEY BO DIDDLEYsong[5][1][1][4]65#9:4 http://localhost:2480/studio/index.html#/database/GratefulDeadConcerts/browse/edit/9:4 265V394originalBERTHAsong[53][1][1][76]65#9:5 http://localhost:2480/studio/index.html#/database/GratefulDeadConcerts/browse/edit/9:5 177V293coverGOING DOWN THE ROAD FEELING BADsong[39][1][1][46]65#9:12 http://localhost:2480/studio/index.html#/database/GratefulDeadConcerts/browse/edit/9:12 167V312originalCASEY JONESsong[47][1][1][33]65#9:13 http://localhost:2480/studio/index.html#/database/GratefulDeadConcerts/browse/edit/9:13 297V423originalDEALsong[80][1][1][65]65#9:17 http://localhost:2480/studio/index.html#/database/GratefulDeadConcerts/browse/edit/9:17 161V328originalSTELLA BLUEsong[31][1][1][46]65#9:21 http://localhost:2480/studio/index.html#/database/GratefulDeadConcerts/browse/edit/9:21 237V241coverCOLD RAIN AND SNOWsong[56][1][1][59]65#9:22 http://localhost:2480/studio/index.html#/database/GratefulDeadConcerts/browse/edit/9:22 225V332originalUNCLE JOHNS BANDsong[49][1][1][60]65#9:28 http://localhost:2480/studio/index.html#/database/GratefulDeadConcerts/browse/edit/9:28 35V234originalMISSISSIPPI HALF-STEPsong[7][1][1][7]65#9:29 http://localhost:2480/studio/index.html#/database/GratefulDeadConcerts/browse/edit/9:29 139V65originalHERE COMES SUNSHINEsong[34][1][1][32]65 The value of $u will always be 65 cause the last record has the performances value of 65. Why is this happening ? I am running OrientDB v1.7.4. Hope someone can help me understand what is happening. If it is a bug of some sort, how can I go around it, and get my output ? -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [orientdb] Upgrade to 2.0.2: Could not initialize class com.orientechnologies.orient.ODatabaseRecordThreadLocal
Hi, Tried again today with 2.0.5 and had the same issue: *java.lang.NoClassDefFoundError: Could not initialize class com.orientechnologies.orient.core.db.ODatabaseRecordThreadLocal* Are there any plans to fix this issue? I'm stuck on 1.7.10 as long as this doesn't work and I really need to upgrade since I think that it will solve another issue we have with this error: *Caused by: com.orientechnologies.common.concur.lock.OLockException: Error on releasing a non acquired lock by the requester 'Thread[OrientDB - BinaryClient (/127.0.0.1:55130),5,OrientDB]' against the resource: '#72:130'* Regards, Roar On Wednesday, February 11, 2015 at 3:41:13 PM UTC+1, Emanuele wrote: Hi, We have already stumbled on this bug, we have an issue here : https://github.com/orientechnologies/orientdb/issues/3501 https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Forientechnologies%2Forientdb%2Fissues%2F3501sa=Dsntz=1usg=AFQjCNHPsGFj1VrAI3OjLa6ntz0mEtynlQ is some problem on class loading with static initialization, tnks for reporting it anyway On 10/02/15 13:32, Roar wrote: I'm testing an upgrade from from 1.7.10 to 2.0.2 using the Migration guide on the Wiki. The only change I had to do make changing the connection pool. We are running OSGI and are embedding all Orientdb bundles and their dependencies into our bundle. I checked that all orientdb bundles included had version 2.0.2 The database is up and running and I tested the url found in the error message from the console. Here is the exception I'm getting. org.apache.felix.log.LogException: com.orientechnologies.orient.core.exception.ODatabaseException: Error on opening database 'remote:localhost:2424/mydb' at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.init(ODatabaseDocumentTx.java:195) at com.orientechnologies.orient.core.db.OPartitionedDatabasePool$DatabaseDocumentTxPolled.init(OPartitionedDatabasePool.java:118) at com.orientechnologies.orient.core.db.OPartitionedDatabasePool$DatabaseDocumentTxPolled.init(OPartitionedDatabasePool.java:114) at com.orientechnologies.orient.core.db.OPartitionedDatabasePool.initQueue(OPartitionedDatabasePool.java:328) at com.orientechnologies.orient.core.db.OPartitionedDatabasePool.init(OPartitionedDatabasePool.java:159) . . Caused by: org.apache.felix.log.LogException: com.orientechnologies.orient.core.exception.ODatabaseException: Error on opening database: localhost:2424/mydb at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.lang.reflect.Constructor.newInstance(Unknown Source) at com.orientechnologies.common.log.OLogManager.error(OLogManager.java:193) at com.orientechnologies.orient.client.remote.OEngineRemote.createStorage(OEngineRemote.java:54) at com.orientechnologies.orient.core.Orient.loadStorage(Orient.java:476) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.init(ODatabaseDocumentTx.java:178) ... 27 more Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.orientechnologies.orient.core.db.ODatabaseRecordThreadLocal at com.orientechnologies.orient.core.record.impl.ODocument.setup(ODocument.java:2193) at com.orientechnologies.orient.core.record.impl.ODocument.init(ODocument.java:107) at com.orientechnologies.orient.client.remote.OStorageRemote.init(OStorageRemote.java:104) at com.orientechnologies.orient.client.remote.OStorageRemote.init(OStorageRemote.java:128) at com.orientechnologies.orient.client.remote.OEngineRemote.createStorage(OEngineRemote.java:47) ... 29 more It looks like the main issue here is 'Could not initialize class com.orientechnologies.orient.core.db.ODatabaseRecordThreadLocal'. And since we are using OSGI and it is very sensitive to classloader changes we suspect that problem may be related to this. BTW. Running the exact same test using the old ODatabaseDocumentPool instead of OPartitionedDatabasePool worked just fine. Any suggestions or fixes that can help are appreciated. Thanks, /Roar -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-databa...@googlegroups.com javascript:. For more options, visit https://groups.google.com/d/optout. -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[orientdb] I can log into my database without any name or password and I'm always root
Well I noticed that I do not even have to give any credentials into the login mask. It *always *gives my *access *to any database. I never mentioned that before. Also if I create a new database via studio and then login without giving anything in. I'm logged in. Is this a usual behaviour and how to stop that? At the end I just want to have one public Account for the database. Using Firefox and latest version of OrientDB. -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [orientdb] Re: vertex as document usage
Hi. I am using the setProperty but I want to keep nested documents in that property like options for the user or accounts that are not just list string but maps. Any nice way of doing this with a vertex? if i use a document object can i create edges for it? because my data is highly connected. Best Regards, Aris Giachnis On Mon, Mar 23, 2015 at 3:06 PM, Colin colinleis...@gmail.com wrote: Hi Aris, A vertex is not a document but does make use of one internally. You can treat a vertex like a document in that you can call setProperty with the same types that ODocument.field() can use. Do note that save() is implicit when using a vertex. If you don't wish to use OrientVertex objects, you can create ODocument objects. To gain access to the underlying Document Database interface via the Graph Database, just call graph.getRawGraph(). Good luck! -Colin Orient Technologies The Company behind OrientDB On Monday, March 23, 2015 at 7:37:06 AM UTC-5, Aris Alexis wrote: hi, sorry for the noob question but i am using the graph database but some vertices i would like them to be documents. for example the user vertex has a lot of sub documents for the location, phones etc. can I do in my code ODocument od=vertex? and then how to save it? can I use the variables then like od.phones[0] or something? do I have to open a new transactional connection to a document database or work with my existing connection for graph? thanks -- --- You received this message because you are subscribed to a topic in the Google Groups OrientDB group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/orient-database/tyiGvLaLtuM/unsubscribe. To unsubscribe from this group and all its topics, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [orientdb] Extreamly slow startup when creating a database (2.0.5 embedded)
Hi Stefan, could you give us additional information about this machine? eg operating system, JVM version Thanks Luigi 2015-03-20 22:36 GMT+01:00 ste...@activitystream.com: Hi, We are seeing the strangest thing here. We have an embedded orentdb (2.0.5 / plocal) that create a new database if missing and set it up property (Vertex and edge types + indexes etc.). This process takes about *10 sec*. on my local machine. When we run this same application on a dedicated-hosted server this process takes more than *6 minutes*. My machine is Intel+SSD but the remote machine is AMD+ shared/mapped disk resources. (Otherwise the machines are pretty identical) We have tested the native/raw disk speed on the dedicated server and it's pretty normal SATA performance (vs. my local SSD). Does anyone know what can be causing this enormous 60 x speed difference? Regards, -Stefan -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [orientdb] Extreamly slow startup when creating a database (2.0.5 embedded)
Hi Luigi, It's Ubuntu 14.04 (LTS) and Java 1.8.0.40. I can give you access to it if it helps. Thank you, -Stefan On Monday, 23 March 2015 07:27:41 UTC, Luigi Dell'Aquila wrote: Hi Stefan, could you give us additional information about this machine? eg operating system, JVM version Thanks Luigi 2015-03-20 22:36 GMT+01:00 ste...@activitystream.com javascript:: Hi, We are seeing the strangest thing here. We have an embedded orentdb (2.0.5 / plocal) that create a new database if missing and set it up property (Vertex and edge types + indexes etc.). This process takes about *10 sec*. on my local machine. When we run this same application on a dedicated-hosted server this process takes more than *6 minutes*. My machine is Intel+SSD but the remote machine is AMD+ shared/mapped disk resources. (Otherwise the machines are pretty identical) We have tested the native/raw disk speed on the dedicated server and it's pretty normal SATA performance (vs. my local SSD). Does anyone know what can be causing this enormous 60 x speed difference? Regards, -Stefan -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-databa...@googlegroups.com javascript:. For more options, visit https://groups.google.com/d/optout. -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[orientdb] Re: OrientDB in(), out(), both() - why do we need direction?
Okay that's perfect. Thanks for the clarification. Definitely need the documents to be updated to help understand the duality of the database and the differences between the two modes. Many times the tutorials/videos will describe an approach and its not clear if they are talking about a vertex or document. The documents elude to vertices and documents being synonymous, but that's clearly not the case in the way the database treats things. On Friday, March 20, 2015 at 4:24:40 PM UTC-4, Colin wrote: Hi Gregory, We're working on the documentation every day to improve it. However, what might be confusing, at first, is that OrientDB is both a document and a graph database. It's a true multi-model database. When treating a class as a document, you can create links manually to other documents, and you're right, the direction is one way. The target document knows nothing about the originator. However, when treating classes as a vertex of a graph (class extends V) and you create edges between them, those relationships are absolutely bi-directional, and you can traverse them in both directions. When creating links for a document, you must handle the relationships yourself. When creating links (edges) for vertices, the system will manage them for you. As an example, if you delete an edge between two vertices, the system will remove the attachments on both vertices. Hope that helps a bit. -Colin Orient Technologies The Company behind OrientDB On Friday, March 20, 2015 at 12:44:10 PM UTC-5, Gregory Pierce wrote: I'm evaluating OrientDB for some particular use cases and while overall the database seems to be very cool (though somewhat confusing in its descriptions as it tries to use the terminology for both document and graph databases... pick one set of terms and only use those), there seem to be some unclear bits. One of the things that I'm having an issue with for OrientDB is that vertices appear to have a direction. Company-Employs-Employee. However an of these vertex relationships are really directionless. To say that a company employs an employee is to also say that an employee is employed_by a company. With the query syntax it appears that to model things properly I would need to create the relationship in each direction when it seems to be something that would make sense to work in both directions. -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[orientdb] Re: vertex as document usage
Hi Aris, A vertex is not a document but does make use of one internally. You can treat a vertex like a document in that you can call setProperty with the same types that ODocument.field() can use. Do note that save() is implicit when using a vertex. If you don't wish to use OrientVertex objects, you can create ODocument objects. To gain access to the underlying Document Database interface via the Graph Database, just call graph.getRawGraph(). Good luck! -Colin Orient Technologies The Company behind OrientDB On Monday, March 23, 2015 at 7:37:06 AM UTC-5, Aris Alexis wrote: hi, sorry for the noob question but i am using the graph database but some vertices i would like them to be documents. for example the user vertex has a lot of sub documents for the location, phones etc. can I do in my code ODocument od=vertex? and then how to save it? can I use the variables then like od.phones[0] or something? do I have to open a new transactional connection to a document database or work with my existing connection for graph? thanks -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [orientdb] Re: Sticky sharding / sharding by sub-graph?
Hi Cho, You could use, let's say, 1,000 shards for your client and provide an hash function that return one of such shard. In this way you can still partitionate your data. Lvc@ On 23 March 2015 at 13:41, Cho Ten Van choten...@gmail.com wrote: oh. I see there is a limit *You can create up to 32,767 (2^15-1) clusters in a database*... http://www.orientechnologies.com/docs/last/Tutorial-Clusters.html so I wouldn't be able to create shards/partitions for 10mln users :( as I understand the only hope for me is to rely on hazelcast to rebalance data closer to that particular app server node which serves particular user... not sure how efficient is it. -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [orientdb] Re: ETL and edges csv
You're looking up Occupation.id, so an index there would be needed. The
rule of the thumb is: create indexes to speedup lookups, like you would do
with a RDBMS.
Lvc@
On 23 March 2015 at 16:45, Gianpaolo Altamura g.altam...@gmail.com wrote:
Hi Luca,
thank you for your repl I have created indexes against Movies.id and
Users.id:
User.json
{
config: {
log: info,
parallel: false
},
source: {
file: {
path: D:\\ETL\\users.dat
}
},
extractor: {
row: {
}
},
transformers: [{
csv: {
separator: ^,
columnsOnFirstLine: false,
columns: [id,
gender:string,
age:integer,
occupationId:integer,
zipCode:string]
}
},
{
vertex: {
class: Users
}
},
{
edge: {
class: hasOccupation,
joinFieldName: occupationId,
lookup: Occupation.id,
unresolvedLinkAction: CREATE
}
}
],
loader: {
orientdb: {
dbURL: remote:localhost/MovieRatings,
dbType: graph,
standardElementConstraints: false,
classes: [{
name: Users,
extends: V
},
{
name: hasOccupation,
extends: E
}],
indexes: [{
class: Users,
fields: [id:integer],
type: UNIQUE
}]
}
}
}
Movies.json
{
config: {
log: info,
parallel: false
},
source: {
file: {
path: D:\\ETL\\movies.dat
}
},
extractor: {
row: {
}
},
transformers: [{
csv: {
separator: ^,
columnsOnFirstLine: false,
columns: [id,
title:string,
genres]
}
},
{
field: {
fieldName: genresArray,
expression: split(genres,'|')
}
},
{
field: {
fieldName: genresArray_0,
expression: genresArray[0]
}
},
{
field: {
fieldName: genresArray_1,
expression: genresArray[1]
}
},
{
field: {
fieldName: genresArray_2,
expression: genresArray[2]
}
},
{
field: {
fieldName: genresArray_3,
expression: genresArray[3]
}
},
{
field: {
fieldName: genresArray_4,
expression: genresArray[4]
}
},
{
field: {
fieldName: genresArray_5,
expression: genresArray[5]
}
},
{
vertex: {
class: Movies
}
},
{
edge: {
class: hasGenera,
joinFieldName: genresArray_0,
lookup: Genres.description,
unresolvedLinkAction: CREATE
}
},
{
edge: {
class: hasGenera,
joinFieldName: genresArray_1,
lookup: Genres.description,
unresolvedLinkAction: CREATE
}
},
{
edge: {
class: hasGenera,
joinFieldName: genresArray_2,
lookup: Genres.description,
unresolvedLinkAction: CREATE
}
},
{
edge: {
class: hasGenera,
joinFieldName: genresArray_3,
lookup: Genres.description,
unresolvedLinkAction: CREATE
}
},
{
edge: {
class: hasGenera,
joinFieldName: genresArray_4,
lookup: Genres.description,
unresolvedLinkAction: CREATE
}
},
{
edge: {
class: hasGenera,
joinFieldName: genresArray_5,
lookup: Genres.description,
unresolvedLinkAction: CREATE
}
}],
loader: {
orientdb: {
dbURL: remote:localhost/MovieRatings,
dbType: graph,
standardElementConstraints: false,
classes: [{
name: Movies,
extends: V
},
{
name: Genres,
extends: V
},
{
name: hasGenera,
extends: E
}],
indexes: [{
class: Movies,
fields: [id:integer],
type: UNIQUE
},
{
class: Genres,
fields: [description:string],
type: UNIQUE_HASH_INDEX
}]
}
}
}
--
---
You received this message because you are subscribed to the Google Groups
OrientDB group.
To unsubscribe from this group and stop receiving emails from it, send an
email to orient-database+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
---
You received this message because you are subscribed to the Google Groups
OrientDB group.
To unsubscribe from this group and stop receiving emails from it, send an email
to orient-database+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[orientdb] ETL and edges csv
I have an csv
ratings.dat (http://files.grouplens.org/datasets/movielens/ml-100k.zip)
that contains this fields:
userID movieID rating
1 1 5
2 1 4
I imported with ETL the user class and movie class now i want to create
with ratings.csv edges (User-Movies with property rating).
How i can use ETL to import this edges?
I have tried with this, but is very very slow and it don't work:
{
config: {
log: debug,
parallel: false
},
source: {
file: {
path: D:\\ETL\\ratings.dat
}
},
extractor: {
row: {
}
},
transformers: [{
csv: {
separator: ^,
columnsOnFirstLine: false,
columns: [userId:integer,
movieId:integer,
rating:integer,
timestamp:datetime]
}
},
{
command: {
command: create edge rated from (select from Users where id
=$input.userId) to (select from Movies where id = $input.movieId) set
ratings = $input.ratings , rating_date = $input.timestamp,
output: edge
}
}],
loader: {
orientdb: {
dbURL: remote:localhost/MovieRatings,
dbType: graph,
standardElementConstraints: false,
classes: [{
name: rated,
extends: E
}]
}
}
}
--
---
You received this message because you are subscribed to the Google Groups
OrientDB group.
To unsubscribe from this group and stop receiving emails from it, send an email
to orient-database+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [orientdb] ETL and edges csv
Hi Gianpaolo,
Have you created indexes against Movies.id and Users.id?
Lvc@
On 23 March 2015 at 16:40, Gianpaolo Altamura g.altam...@gmail.com wrote:
I have an csv
ratings.dat (http://files.grouplens.org/datasets/movielens/ml-100k.zip)
that contains this fields:
userID movieID rating
1 1 5
2 1 4
I imported with ETL the user class and movie class now i want to create
with ratings.csv edges (User-Movies with property rating).
How i can use ETL to import this edges?
I have tried with this, but is very very slow and it don't work:
{
config: {
log: debug,
parallel: false
},
source: {
file: {
path: D:\\ETL\\ratings.dat
}
},
extractor: {
row: {
}
},
transformers: [{
csv: {
separator: ^,
columnsOnFirstLine: false,
columns: [userId:integer,
movieId:integer,
rating:integer,
timestamp:datetime]
}
},
{
command: {
command: create edge rated from (select from Users where id
=$input.userId) to (select from Movies where id = $input.movieId) set
ratings = $input.ratings , rating_date = $input.timestamp,
output: edge
}
}],
loader: {
orientdb: {
dbURL: remote:localhost/MovieRatings,
dbType: graph,
standardElementConstraints: false,
classes: [{
name: rated,
extends: E
}]
}
}
}
--
---
You received this message because you are subscribed to the Google Groups
OrientDB group.
To unsubscribe from this group and stop receiving emails from it, send an
email to orient-database+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
---
You received this message because you are subscribed to the Google Groups
OrientDB group.
To unsubscribe from this group and stop receiving emails from it, send an email
to orient-database+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[orientdb] Re: ETL and edges csv
Hi Luca,
thank you for your repl I have created indexes against Movies.id and
Users.id:
User.json
{
config: {
log: info,
parallel: false
},
source: {
file: {
path: D:\\ETL\\users.dat
}
},
extractor: {
row: {
}
},
transformers: [{
csv: {
separator: ^,
columnsOnFirstLine: false,
columns: [id,
gender:string,
age:integer,
occupationId:integer,
zipCode:string]
}
},
{
vertex: {
class: Users
}
},
{
edge: {
class: hasOccupation,
joinFieldName: occupationId,
lookup: Occupation.id,
unresolvedLinkAction: CREATE
}
}
],
loader: {
orientdb: {
dbURL: remote:localhost/MovieRatings,
dbType: graph,
standardElementConstraints: false,
classes: [{
name: Users,
extends: V
},
{
name: hasOccupation,
extends: E
}],
indexes: [{
class: Users,
fields: [id:integer],
type: UNIQUE
}]
}
}
}
Movies.json
{
config: {
log: info,
parallel: false
},
source: {
file: {
path: D:\\ETL\\movies.dat
}
},
extractor: {
row: {
}
},
transformers: [{
csv: {
separator: ^,
columnsOnFirstLine: false,
columns: [id,
title:string,
genres]
}
},
{
field: {
fieldName: genresArray,
expression: split(genres,'|')
}
},
{
field: {
fieldName: genresArray_0,
expression: genresArray[0]
}
},
{
field: {
fieldName: genresArray_1,
expression: genresArray[1]
}
},
{
field: {
fieldName: genresArray_2,
expression: genresArray[2]
}
},
{
field: {
fieldName: genresArray_3,
expression: genresArray[3]
}
},
{
field: {
fieldName: genresArray_4,
expression: genresArray[4]
}
},
{
field: {
fieldName: genresArray_5,
expression: genresArray[5]
}
},
{
vertex: {
class: Movies
}
},
{
edge: {
class: hasGenera,
joinFieldName: genresArray_0,
lookup: Genres.description,
unresolvedLinkAction: CREATE
}
},
{
edge: {
class: hasGenera,
joinFieldName: genresArray_1,
lookup: Genres.description,
unresolvedLinkAction: CREATE
}
},
{
edge: {
class: hasGenera,
joinFieldName: genresArray_2,
lookup: Genres.description,
unresolvedLinkAction: CREATE
}
},
{
edge: {
class: hasGenera,
joinFieldName: genresArray_3,
lookup: Genres.description,
unresolvedLinkAction: CREATE
}
},
{
edge: {
class: hasGenera,
joinFieldName: genresArray_4,
lookup: Genres.description,
unresolvedLinkAction: CREATE
}
},
{
edge: {
class: hasGenera,
joinFieldName: genresArray_5,
lookup: Genres.description,
unresolvedLinkAction: CREATE
}
}],
loader: {
orientdb: {
dbURL: remote:localhost/MovieRatings,
dbType: graph,
standardElementConstraints: false,
classes: [{
name: Movies,
extends: V
},
{
name: Genres,
extends: V
},
{
name: hasGenera,
extends: E
}],
indexes: [{
class: Movies,
fields: [id:integer],
type: UNIQUE
},
{
class: Genres,
fields: [description:string],
type: UNIQUE_HASH_INDEX
}]
}
}
}
--
---
You received this message because you are subscribed to the Google Groups
OrientDB group.
To unsubscribe from this group and stop receiving emails from it, send an email
to orient-database+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[orientdb] Re: ETL and edges csv
Hi Luca, when I created Occupation class I have created an index on Occupation.id . *[58:command] DEBUG executed command=sql.create edge rated from (select from Users where id = $input.userId) to (select from Movies where id = $input.movieId) set ratings = $input.ratings , rating_date = $input.timestamp, result=[]* I think the problem is on $input.userId and $input.movies variables... because create edge instruction don't create any edge. Can you help me? -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[orientdb] Re: ETL and edges csv
This is the output of debug console:
[58:csv] DEBUG Transformer input: 2^648^4^978299913
[58:csv] DEBUG parsing=2^648^4^978299913
[58:csv] DEBUG document={userId:2,movieId:648,rating:4,timestamp:Mon Jan 12
08:44:59 CET 1970}
[58:csv] DEBUG Transformer output:
{userId:2,movieId:648,rating:4,timestamp:MonJan 12 08:44:59 CET 1970}
[58:command] DEBUG Transformer input:
{userId:2,movieId:648,rating:4,timestamp:Mon Jan 12 08:44:59 CET 1970}
[58:command] DEBUG executing command=create edge rated from (select from
Users where id = $input.userId) to (select from Movies where id =
$input.movieId) set ratings = $input.ratings , rating_date =
$input.timestamp...
[58:command] DEBUG executed command=sql.create edge rated from (select from
Users where id = $input.userId) to (select from Movies where id =
$input.movieId) set ratings = $input.ratings , rating_date =
$input.timestamp, result=[]
[58:command] DEBUG Transformer output (same as input):
{userId:2,movieId:648,rating:4,timestamp:Mon Jan 12 08:44:59 CET 1970}
--
---
You received this message because you are subscribed to the Google Groups
OrientDB group.
To unsubscribe from this group and stop receiving emails from it, send an email
to orient-database+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[orientdb] Re: ETL and edges csv
Solved! The problem are the variables. I changed $input.attribute in
${input.attribute}
Ratings.json
{
config: {
log: info,
parallel: false
},
source: {
file: {
path: D:\\ETL\\ratings.dat
}
},
extractor: {
row: {
}
},
transformers: [{
csv: {
separator: ^,
columnsOnFirstLine: false,
columns: [userId:integer,
movieId:integer,
rating:integer,
timestamp:datetime]
}
},
{
command: {
command: create edge rated from (select from Users where id =
${input.userId}) to (select from Movies where id = ${input.movieId}) set
ratings = ${input.ratings} , rating_date = ${input.timestamp},
output: edge
}
}],
loader: {
orientdb: {
dbURL: remote:localhost/MovieRatings,
dbType: graph,
standardElementConstraints: false,
classes: [{
name: rated,
extends: E
}]
}
}
}
--
---
You received this message because you are subscribed to the Google Groups
OrientDB group.
To unsubscribe from this group and stop receiving emails from it, send an email
to orient-database+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [orientdb] Re: vertex as document usage
When you say nested documents, do you mean like a map or set of embedded or linked documents? You can use all the same types in a vertex as you can with a document. Just think of an OrientVertex as a hybrid of a node and a document. Just stick with the Vertex and create edges for it while setting its properties like a document. -Colin Orient Technologies The Company behind OrientDB On Monday, March 23, 2015 at 10:27:04 AM UTC-5, Aris Alexis wrote: Hi. I am using the setProperty but I want to keep nested documents in that property like options for the user or accounts that are not just list string but maps. Any nice way of doing this with a vertex? if i use a document object can i create edges for it? because my data is highly connected. Best Regards, Aris Giachnis On Mon, Mar 23, 2015 at 3:06 PM, Colin colinl...@gmail.com javascript: wrote: Hi Aris, A vertex is not a document but does make use of one internally. You can treat a vertex like a document in that you can call setProperty with the same types that ODocument.field() can use. Do note that save() is implicit when using a vertex. If you don't wish to use OrientVertex objects, you can create ODocument objects. To gain access to the underlying Document Database interface via the Graph Database, just call graph.getRawGraph(). Good luck! -Colin Orient Technologies The Company behind OrientDB On Monday, March 23, 2015 at 7:37:06 AM UTC-5, Aris Alexis wrote: hi, sorry for the noob question but i am using the graph database but some vertices i would like them to be documents. for example the user vertex has a lot of sub documents for the location, phones etc. can I do in my code ODocument od=vertex? and then how to save it? can I use the variables then like od.phones[0] or something? do I have to open a new transactional connection to a document database or work with my existing connection for graph? thanks -- --- You received this message because you are subscribed to a topic in the Google Groups OrientDB group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/orient-database/tyiGvLaLtuM/unsubscribe . To unsubscribe from this group and all its topics, send an email to orient-databa...@googlegroups.com javascript:. For more options, visit https://groups.google.com/d/optout. -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[orientdb] Re: OrientDB in(), out(), both() - why do we need direction?
Agreed! That's something we're working on to improve the documentation and the tutorials to be more clear. Thanks, -Colin Orient Technologies The Company behind OrientDB On Monday, March 23, 2015 at 9:40:12 AM UTC-5, Gregory Pierce wrote: Okay that's perfect. Thanks for the clarification. Definitely need the documents to be updated to help understand the duality of the database and the differences between the two modes. Many times the tutorials/videos will describe an approach and its not clear if they are talking about a vertex or document. The documents elude to vertices and documents being synonymous, but that's clearly not the case in the way the database treats things. On Friday, March 20, 2015 at 4:24:40 PM UTC-4, Colin wrote: Hi Gregory, We're working on the documentation every day to improve it. However, what might be confusing, at first, is that OrientDB is both a document and a graph database. It's a true multi-model database. When treating a class as a document, you can create links manually to other documents, and you're right, the direction is one way. The target document knows nothing about the originator. However, when treating classes as a vertex of a graph (class extends V) and you create edges between them, those relationships are absolutely bi-directional, and you can traverse them in both directions. When creating links for a document, you must handle the relationships yourself. When creating links (edges) for vertices, the system will manage them for you. As an example, if you delete an edge between two vertices, the system will remove the attachments on both vertices. Hope that helps a bit. -Colin Orient Technologies The Company behind OrientDB On Friday, March 20, 2015 at 12:44:10 PM UTC-5, Gregory Pierce wrote: I'm evaluating OrientDB for some particular use cases and while overall the database seems to be very cool (though somewhat confusing in its descriptions as it tries to use the terminology for both document and graph databases... pick one set of terms and only use those), there seem to be some unclear bits. One of the things that I'm having an issue with for OrientDB is that vertices appear to have a direction. Company-Employs-Employee. However an of these vertex relationships are really directionless. To say that a company employs an employee is to also say that an employee is employed_by a company. With the query syntax it appears that to model things properly I would need to create the relationship in each direction when it seems to be something that would make sense to work in both directions. -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [orientdb] Re: Sticky sharding / sharding by sub-graph?
Lvc@ thank you for your reply, I will consider this option :) On Monday, March 23, 2015 at 6:44:05 PM UTC+3, Lvc@ wrote: Hi Cho, You could use, let's say, 1,000 shards for your client and provide an hash function that return one of such shard. In this way you can still partitionate your data. Lvc@ -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [orientdb] Re: insert or select?
Hi, I already answered in your previous email ;-) 2015-03-22 0:44 GMT+01:00 pumplerod pumple...@gmail.com: I'm sorry, I thought I was in the pyorient group. Though perhaps someone here can answer the question. On Saturday, March 21, 2015 at 6:43:21 PM UTC-5, pumplerod wrote: is there a way to select a record or insert if it does not exist? The doc for OrientDB mentions upsert, however I cannot seem to get that to work even through the OrientDB web portal. I have a class, Locale which I would like to select in order to create an edge with another vertex class. If the Locale does not already exist I would like to create and return it. It should always be unique. -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [orientdb] find_or_create in OrientDB?
Hi
Upsert is just a keyword in the UPDATE statement, eg.
update V set id = '12', name = 'foo' upsert where id = '12'
you can find all the docs here
http://www.orientechnologies.com/docs/last/SQL-Update.html
Luigi
2015-03-22 0:17 GMT+01:00 pumplerod pumple...@gmail.com:
Is there example usage of this? When I try and use upsert all I get is an
error.
I have a class Locale which I only want to add if the specified locale
does not already exist...
upsert into Locale set type='state' name='TX' where type='state' name='TX'
The error states that there is no command executor for command request:
sql.upsert
On Wednesday, February 4, 2015 at 9:08:04 AM UTC-6, Luigi Dell'Aquila
wrote:
Hi Artem,
have you tried with UPSERT?
Here you can find all the references:
http://www.orientechnologies.com/docs/last/orientdb.wiki/SQL-Update.html
Regards
Luigi
2015-02-04 13:25 GMT+01:00 Artem Shoobovych shybo...@gmail.com:
Hi there!
Lately, I've been playing with OrientDB of 1.7.10 and now 2.0.1. And
currently I am stuck with poor SQL syntax or my misunderstanding of it.
What I want is to create an instance of a class if it does not exist. I
tried different solutions, like `ifnull`, `first` and `if`. But it seems to
work only on existing values.
For example, let us have a class `City` with only one instance -
`{name: Lisboa}`.
These do not work:
select if(count(select City where name = Krakow) 1, null, (insert
into City set name = Krakow return @rid))
select ifnull((select @rid from City where name = Krakow), (insert
into City set name = Krakow return @rid))
select first((select @rid from City where name = Krakow), (insert
into City set name = Krakow))
I believe I need some kind of `exists` operator:
select if( exists(select from City where name = Krakow), (select
@rid from City where name = Krakow), (insert into City set name =
Krakow))
Are there any possibility for this except stored functions?
Thanks!
--
---
You received this message because you are subscribed to the Google
Groups OrientDB group.
To unsubscribe from this group and stop receiving emails from it, send
an email to orient-databa...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
---
You received this message because you are subscribed to the Google Groups
OrientDB group.
To unsubscribe from this group and stop receiving emails from it, send an
email to orient-database+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
---
You received this message because you are subscribed to the Google Groups
OrientDB group.
To unsubscribe from this group and stop receiving emails from it, send an email
to orient-database+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [orientdb] Strip Javascript and/or HTML code when inserting/updating
Hi Stas, you could implement a hook for this. You can do this in Java or Javascript (see http://www.orientechnologies.com/docs/last/Hook.html ), just ensure to * bind your application logic on the following events: onBeforeCreate, onBeforeUpdate * let your hook functions return RECORD_CHANGED Luigi 2015-03-22 0:27 GMT+01:00 Stas Dmitrenko stanislavski...@gmail.com: Hello. I'm new here. So could you please help? I want to strip all html characters from text before inserting to DB. Is there a way to do that in OrientDB? It can work like this: OrientDB got call with Insert or Update. Then it replaces all html characters with safe equivalent and saves it (or just delete all this staff). Done) How to do that? -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [orientdb] Re: Is using ElasticSearch with OrientDB possible?
Thanks all for the suggestions. From what I've understood so far, I think it's better to go with Orientdb-Lucene for now. I don't think its worth the effort to make ES work with OrientDB as it clearly requires some additional work in updating the indices in ES, since the application I am working on is not going to have millions of users. I have used neither ES nor Lucene before, but I guess Lucene should be enough since ES is built on top of Lucene. So I guess I'll give it a try. On Fri, Mar 20, 2015 at 5:00 PM, Nicolas Harraudeau nicolas.harraud...@gmail.com wrote: Hi Kevin, There might be a way to mark documents as updated. This is not an easy solution and I didn't try it yet. It uses MVCC and Optimistic Transactions (you can read more about this here http://www.orientechnologies.com/docs/2.0/orientdb.wiki/Transactions.html ). Let's say you have your application on one side which is adding, deleting, updating documents in OrientDB. On the other side you have your replication process which reads OrientDB and writes in Elasticsearch. When your replication process starts scanning OrientDB, it creates/replace first a unique vertex (let's call it checkpoint vertex) which contains the start date of the scan. Each time your application modifies OrientDB, it reads the checkpoint vertex and set the modification date of each indexed vertex/edge to its date. If a scan started during the modification, the checkpoint vertex has been changed and the transaction should fail. For deletes, a vertex describing the delete has to be created. This has some drawbacks: - the application has either to know what is indexed in ES, or it has to set a date on every vertex/edge. - you must use transactions even when you want to modify one vertex/edge. I don't like this solution very much but it might be ok for you. You might also use a file or something else as a modifications log. But then you can't backup both the modification log and the OrientDB graph at the same time. Regards, On Thursday, March 19, 2015 at 5:37:18 PM UTC+1, Nicolas Harraudeau wrote: Hi Patrick, I have searched a way to do it myself but didn't found a correct way to do it. Here is what I found: Having worked with indexing problems before on another search engine and other sources, there are always two different jobs: - The first one does a full scan of the source. With OrientDB it is possible using a simple JDBC driver and a few requests. OrientDB can be completely scanned using pagination http://www. orientechnologies.com/docs/last/Pagination.html - The second job is more complex. It has to fetch only modified documents as often as you need in order to have up to date results. When fetching updates you want to scan from the start date of the last scan because modifications can happen during the scan itself. Let's name this start date checkpoint. My first thought was that I could save the last modification timestamp in OrientDB docs. But I didn't found any way to generate it during commit. It MUST not be generated by the application as this would add dates which are generated BEFORE the checkpoint but saved AFTER this same checkpoint. Think of your application making a modification that spans the start of the update scan. The second approach would be to create a Modifications to scan vertex and link to it every modified document. This would not scale as it would conflict more and more during transactions. The third approach is to use Hooks which would mark documents as modified. However the documentation is rather poor on those. In order to be used by an update scan, hook registration need to be transactional. I asked here if adding a hook invalidates the running transactions ( https://groups.google.com/forum/#!topic/orient-database/FBHiZg68b1s) but did not receive any answer. I tested it myself and found that it is not working as I would like (https://github.com/orientechnologies/orientdb/ issues/3763). There is still no information as to how it SHOULd work. No specifications. Maybe one of those features will enable to have a correct update stream: https://github.com/orientechnologies/orientdb/issues/2652 https://github.com/orientechnologies/orientdb/issues/1227 In the mean time, I don't see any way to index correctly OrientDB. If someone succeeded at indexing OrientDB I am interested too. OrientDB-Lucene is promising but it is too limited for me right now. I cannot work without features like highlights or complex scoring. On Monday, March 16, 2015 at 4:41:36 PM UTC+1, Kevin I wrote: I can see that OrientDB lucene indices can be done through orientdb-lucene https://github.com/orientechnologies/orientdb-lucene, but is there a way to use ElasticSearch in OrientDB? In TitanDB, ElasticSearch support was inbuilt. It would be great if OrientDB has that too. If not, can I make the two work together out of the box? I haven't used ElasticSearch before, so it would be of
[orientdb] EXPLAIN for gremlin
1) Is there any way to get query explanations for gremlin queries like EXPLAIN does for osql? 2) My understanding is that gremlin queries are translated into osql queries, is there any way to see this translation? It would be helpful for performance tuning and learning osql. -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
