Oh, OK, thanks for the clarification. In the meantime I played with the propstat tool and shows in my case 50% reduction in DB size (and will probably show a lot in performance). Very promising ... ;-) --- balazs
On 3/7/11 3:19 PM, Tobias Ivarsson wrote: > That class is in Neo4j kernel. It is the actual implementation class. > You can find the source code here: > https://github.com/neo4j/graphdb/blob/master/kernel/src/main/java/org/neo4j/kernel/impl/nioneo/store/ShortString.java > > All of the classes in the org.neo4j.kernel.impl.nioneo.store package of my > neo4j-admin-store project are of that kind, facade classes that provide > access to internals of the neo4j kernel implementation. > > Cheers, > Tobias > > On Mon, Mar 7, 2011 at 1:00 PM, Balazs E. Pataki<[email protected]>wrote: > >> Hi Tobias, >> >> I tried to use ShortStringEncoding.java, however it seems to refer to a >> class called ShortString, which is not available in github: >> >> >> https://github.com/thobe/neo4j-admin-store/tree/master/src/main/java/org/neo4j/kernel/impl/nioneo/store >> >> Could you also please provide it? >> >> Thanks, >> --- >> balazs >> >> On 3/4/11 5:05 PM, Tobias Ivarsson wrote: >>> Hi Balazs, >>> >>> I have created such a thing, the tool is available here: >>> >> https://github.com/downloads/thobe/neo4j-admin-store/propstat-1.3.M03.jar >>> Run it as: java -jar propstat-1.3.M03.jar<path to your neo4j store> >>> >>> This tool will gather statistics about all of your properties, and output >>> how many you have of each kind. >>> For short string properties it will output how many you have in each >>> encoding. >>> For regular string properties it will output how many could have been >> stored >>> as short strings, and with which encoding that would be. >>> >>> That jar file contains a class that can be used for testing if a given >>> string is "short" or not, and how it would be stored if it was considered >>> short: >>> >> https://github.com/thobe/neo4j-admin-store/blob/master/src/main/java/org/neo4j/kernel/impl/nioneo/store/ShortStringEncoding.java >>> The method you are interested in is (the first one) getEncoding(String), >> it >>> returns one of the Enum constants in that class, or null if the provided >>> string would not be considered "short". >>> >>> While I wrote this I realized that it would be really simple to write a >> tool >>> that converts all strings that are short enough to be stored as short >>> strings, that tool is available here: >>> >> https://github.com/downloads/thobe/neo4j-admin-store/makeshort-1.3.M03.jar >>> Run it as: java -jar makeshort-1.3.M03.jar<path to your neo4j store> >>> >>> This tool will modify your string properties to store them as short >> strings >>> when possible. It will then rebuild the id generator for the >>> DynamicStringStore so that the "holes" created where the strings were >>> previously stored will be reused the next time you write string >> properties >>> to Neo4j (unless they are also short of course). >>> >>> Please note that both of these tools come with no official support or >>> guarantees. Use at your own risk. >>> >>> Cheers, >>> Tobias >>> >>> On Fri, Mar 4, 2011 at 10:19 AM, Balazs E. Pataki<[email protected] >>> wrote: >>> >>>> Actually both options would be very useful, but the first option (string >>>> as input and tell me if it is short or not) would be enough for me for >>>> now if available. >>>> >>>> Thanks, >>>> --- >>>> balazs >>>> >>>> On 3/3/11 6:37 PM, Tobias Ivarsson wrote: >>>>> Yes, the statistics tool almost does that, but with a wider range of >>>>> possible encodings. >>>>> >>>>> What would you want from a class like that? Give it a string as input >> and >>>>> have it tell you if it is short or not? Or go through an existing store >>>> and >>>>> tell you how many of the strings are short? >>>>> >>>>> -tobias >>>>> >>>>> On Thu, Mar 3, 2011 at 5:18 PM, Balazs E. Pataki<[email protected] >>>>> wrote: >>>>> >>>>>> Hi Tobias, >>>>>> >>>>>> Do you happen to have some little class which can at tell us which >>>>>> property values neo4j would consider short and which of your 6 >> encoding >>>>>> would it use? Maybe the string statistics collecting program you >> created >>>>>> earlier has this logic - but I'm just guessing. >>>>>> >>>>>> Thanks, >>>>>> --- >>>>>> balazs >>>>>> >>>>>> On 3/3/11 11:52 AM, Tobias Ivarsson wrote: >>>>>>> No there is no simpler way, yet. We've been thinking about creating a >>>>>> "short >>>>>>> string compression tool" for accomplishing this, but haven't done so >>>> yet. >>>>>>> >>>>>>> Cheers, >>>>>>> Tobias >>>>>>> >>>>>>> On Thu, Mar 3, 2011 at 11:35 AM, Balazs E. Pataki< >> [email protected] >>>>>>> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I have a big database based on Neo4J 1.2. Now, if I would like to >> use >>>>>>>> the "short strings" feature of Neo4j 1.3 M03 I should regenerate my >>>> full >>>>>>>> database, that is all strings should be reset so that it may or may >>>> not >>>>>>>> be stored according to the new "short strings" policy. >>>>>>>> >>>>>>>> It seems to me that the easiest way to do this would be to somehow >> be >>>>>>>> able to copy the full 1.2 database to a newly created 1.3 M03 >> database >>>>>>>> by traversing the 1.2 database. But there maybe a simpler (neo4j >>>>>>>> builtin) way to do this. Any hints about this? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> --- >>>>>>>> balazs >>>>>>>> _______________________________________________ >>>>>>>> Neo4j mailing list >>>>>>>> [email protected] >>>>>>>> https://lists.neo4j.org/mailman/listinfo/user >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> _______________________________________________ >>>>>> Neo4j mailing list >>>>>> [email protected] >>>>>> https://lists.neo4j.org/mailman/listinfo/user >>>>>> >>>>> >>>>> >>>>> >>>> _______________________________________________ >>>> Neo4j mailing list >>>> [email protected] >>>> https://lists.neo4j.org/mailman/listinfo/user >>>> >>> >>> >>> >> _______________________________________________ >> Neo4j mailing list >> [email protected] >> https://lists.neo4j.org/mailman/listinfo/user >> > > > _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
