Hmmm, I had appended the logs and a screen shot of what I did - did
something strip it out?
Trying again:
Ofbiz.Log snippet from time frame
=======================================================================
2010-12-01 15:15:19,937 (http-0.0.0.0-8443-7) [
ControlServlet.java:327:INFO ] [[[login] Request Done-
total:0.073,since last([login] Request B...):0.073]]
2010-12-01 15:15:19,941 (http-0.0.0.0-8443-7) [
ControlServlet.java:144:INFO ] [[[viewdatafile] Request Begun,
encoding=[UTF-8]- total:0.0,since last(Begin):0.0]]
2010-12-01 15:15:19,948 (http-0.0.0.0-8443-7) [
RequestHandler.java:740:INFO ] Rendering View [viewdatafile],
sessionId=C4503F3026169883A29BCA0B35E92EEF.jvm1
2010-12-01 15:15:19,982 (http-0.0.0.0-8443-7) [
ScreenFactory.java:128:INFO ] Got 2 screens in 0.0070s from:
file:/C:/ofbiz/mysqlrelease10.04/framework/webtools/widget/MiscScreens.xml
2010-12-01 15:15:20,233 (http-0.0.0.0-8443-7) [
ServerHitBin.java:627:INFO ] Visit delegatorName=default,
ServerHitBin delegatorName=default
2010-12-01 15:15:20,235 (http-0.0.0.0-8443-7) [
ControlServlet.java:327:INFO ] [[[viewdatafile] Request Done-
total:0.294,since last([viewdatafile] Re...):0.294]]
2010-12-01 15:15:24,219 (http-0.0.0.0-8443-5) [
ContextFilter.java:201:INFO ] [Request]: /webtools/control/viewdatafile
2010-12-01 15:15:24,219 (http-0.0.0.0-8443-5) [
ControlServlet.java:144:INFO ] [[[viewdatafile] Request Begun,
encoding=[UTF-8]- total:0.0,since last(Begin):0.0]]
2010-12-01 15:15:24,229 (http-0.0.0.0-8443-5) [
ConfigXMLReader.java:120:INFO ] controller loaded: 0.0s, 23 requests,
16 views in
file:/C:/ofbiz/mysqlrelease10.04/framework/common/webcommon/WEB-INF/common-controller.xml
2010-12-01 15:15:24,233 (http-0.0.0.0-8443-5) [
ConfigXMLReader.java:120:INFO ] controller loaded: 0.0s, 5 requests,
0 views in
file:/C:/ofbiz/mysqlrelease10.04/framework/common/webcommon/WEB-INF/tempexpr-controller.xml
2010-12-01 15:15:24,237 (http-0.0.0.0-8443-5) [
ConfigXMLReader.java:120:INFO ] controller loaded: 0.0010s, 8
requests, 2 views in
file:/C:/ofbiz/mysqlrelease10.04/framework/testtools/webapp/testtools/WEB-INF/controller.xml
2010-12-01 15:15:24,238 (http-0.0.0.0-8443-5) [
ConfigXMLReader.java:120:INFO ] controller loaded: 0.013s, 110
requests, 64 views in
file:/C:/ofbiz/mysqlrelease10.04/framework/webtools/webapp/webtools/WEB-INF/controller.xml
2010-12-01 15:15:24,246 (http-0.0.0.0-8443-5) [
RequestHandler.java:740:INFO ] Rendering View [viewdatafile],
sessionId=C4503F3026169883A29BCA0B35E92EEF.jvm1
2010-12-01 15:15:24,294 (http-0.0.0.0-8443-5)
[ModelDataFileReader.java:65 :INFO ]
[ModelDataFileReader.getModelDataFileReader] : creating reader.
2010-12-01 15:15:24,348 (http-0.0.0.0-8443-5) [
ScreenFactory.java:128:INFO ] Got 10 screens in 0.0070s from:
file:/C:/ofbiz/mysqlrelease10.04/framework/webtools/widget/CommonScreens.xml
2010-12-01 15:15:24,358 (http-0.0.0.0-8443-5) [
ScreenFactory.java:128:INFO ] Got 22 screens in 0.0090s from:
file:/C:/ofbiz/mysqlrelease10.04/framework/common/widget/CommonScreens.xml
2010-12-01 15:15:24,512 (http-0.0.0.0-8443-5) [
ServerHitBin.java:627:INFO ] Visit delegatorName=default,
ServerHitBin delegatorName=default
2010-12-01 15:15:24,515 (http-0.0.0.0-8443-5) [
ControlServlet.java:327:INFO ] [[[viewdatafile] Request Done-
total:0.296,since last([viewdatafile] Re...):0.296]]
2010-12-01 15:15:29,084 (http-0.0.0.0-8443-5) [
ContextFilter.java:201:INFO ] [Request]: /webtools/control/viewdatafile
2010-12-01 15:15:29,085 (http-0.0.0.0-8443-5) [
ControlServlet.java:144:INFO ] [[[viewdatafile] Request Begun,
encoding=[UTF-8]- total:0.0,since last(Begin):0.0]]
2010-12-01 15:15:29,092 (http-0.0.0.0-8443-5) [
RequestHandler.java:740:INFO ] Rendering View [viewdatafile],
sessionId=C4503F3026169883A29BCA0B35E92EEF.jvm1
2010-12-01 15:15:29,134 (http-0.0.0.0-8443-5)
[ModelDataFileReader.java:65 :INFO ]
[ModelDataFileReader.getModelDataFileReader] : creating reader.
2010-12-01 15:15:29,353 (http-0.0.0.0-8443-5) [
ServerHitBin.java:627:INFO ] Visit delegatorName=default,
ServerHitBin delegatorName=default
2010-12-01 15:15:29,357 (http-0.0.0.0-8443-5) [
ControlServlet.java:327:INFO ] [[[viewdatafile] Request Done-
total:0.272,since last([viewdatafile] Re...):0.272]]
2010-12-01
=====================================================================================
STeps:
I went to the Webtools / Work with Datafiles screen
I entered the following string in the "Definition Filename or URL"
box. The file is attached to the email
"c:\files\tofino\import data\tofinodata.xml"
I selected the "Run" button
Nothing happened and I can't see anything in the ofbiz.log which
would point to why the system is not accepting the XML file
Thanks
James
-----Original Message-----
From: BJ Freeman [mailto:[email protected]]
Sent: December-01-10 3:41 PM
To: [email protected]
Subject: [SPAM]Re: How to import a pre existing product database
sorry see no logs and not enough detail.
I have no way to replicated your steps.
=========================
BJ Freeman
Strategic Power Office with Supplier
Automation<http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com<http://www.specialtymarket.com/> Systems
Integrator-- Glad to Assist
Chat Y! messenger: bjfr33man
James Lawton sent the following on 12/1/2010 12:23 PM:
Here is the section of ofbiz.log for the timeframe
Here is what I did:
entered the file information and pressed "Run"
Thanks
James
-----Original Message-----
From: BJ Freeman [mailto:[email protected]]
Sent: December-01-10 2:18 PM
To: [email protected]
Subject: [SPAM]Re: How to import a pre existing product database
take a look at your logs.
please give detail steps
=========================
BJ Freeman
Strategic Power Office with Supplier Automation<
<http://www.businessesnetwork.com/automation/viewforum.php?f=52>
http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com< <http://www.specialtymarket.com/>
http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
Chat Y! messenger: bjfr33man
James Lawton sent the following on 12/1/2010 10:27 AM:
Thanks for the link and the record information...
I have altered the fields to fit the product record field names and
tried to use the "work with data files" tool again
I enter the Definition File Name in the screen
"<file:///\\locahost\c:\files\tofino\import%20data\tofinodata.xml>
file://locahost/c:\files\tofino\import data\tofinodata.xml" and
select the URL check box and then hit Run. The screen flickers but
nothing happens. The instructions say that a drop down box will
appear with a list of the definitions to pick from...
Thoughts?
Here is the modified definition file:
<data-file name="c:\files\tofino\import
data\itemdatabasereportmod.txt" separator-style="delimited"
type-code="UTF-8"
delimiter="	" text-delimiter=""
description="this takes the product data from jeffs file"
has-header="true">
<record name="Product">
<field name="manufacturerPartyId" ignored="false"
type="String"/>
<field name="productId" ignored="false" type="String"/>
<field name="primaryProductCategoryId" expression="true"
default-value="tools" type="String"/>
<field name="productTypeId" expression="true"
default-value="FINISHED_GOOD" type="String"/>
<field name="productName" ignored="false" type="String"/>
<field name="description" ignored="false" type="String"/>
<field name="quantityIncluded" expression="true" default-value="1"
type="String"/>
<field name="priceDetailText" ignored="true"
type="String"/>
</record>
</data-file>
Thanks
James
-----Original Message-----
From: BJ Freeman [mailto:[email protected]]
Sent: December-01-10 12:37 PM
To: [email protected]
Subject: [SPAM]Re: How to import a pre existing product database
if you look at the table the manufacture field is in then look up the
same in the entities
<https://demo-trunk.ofbiz.apache.org/webtools/control/entitymaint>
https://demo-trunk.ofbiz.apache.org/webtools/control/entitymaint
Just to clarify
your file has a Record=Product.
this means you will be creating a Product entity file, so only
fields in the product entity can be created.
<https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?ent
it>
https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?enti
t
yName=Product&find=true&VIEW_SIZE=50&VIEW_INDEX=0
if you have data that goes in different entities then you need to
create a Record for each entity with its fields.
James Lawton sent the following on 12/1/2010 9:16 AM:
=========================
BJ Freeman
Strategic Power Office with Supplier
Automation<http://www.businessesnetwork.com/automation/viewforum.php?
f
=52> Specialtymarket.com< <http://www.specialtymarket.com/>
http://www.specialtymarket.com/> Systems
Integrator-- Glad to Assist
Chat Y! messenger: bjfr33man
Thanks
I must be missing the spot in web tools it has a complete list of
the field names - could you point that out again? For example what
is the right field name for the manufacturer / supplier? I did
take that out the db....
Thanks
James
-----Original Message-----
From: BJ Freeman [mailto:[email protected]]
Sent: November-30-10 11:42 PM
To: [email protected]
Subject: [SPAM]Re: How to import a pre existing product database
there should be more to the log that would pinpoint what is going on.
I am guessing it is your input to the form, since this has not
been changed 09.
if you look at the file mentioned you will see
<script
location="component://webtools/webapp/webtools/WEB-INF/actions/dataf
i
l
e/viewdatafile.groovy"/>
that is where the error referenced.
your xml(not xsd) should have field names as you see them in
webtools.
when you have underscores like manufacturer_party_id then that is
what is in the db but ofbiz will not recognize it in Webtools or
code.
James Lawton sent the following on 11/30/2010 7:34 PM:
I think I am getting closer - thanks for the tips and the patience!
I have been through all the files and here is what I have gotten
when running my definition file through the data tools
org.ofbiz.widget.screen.ScreenRenderException: Error rendering
screen
[component://webtools/widget/MiscScreens.xml#viewdatafile]:
groovy.lang.MissingPropertyException: No such property:
definitionUrl for class: viewdatafile (No such property:
definitionUrl for class:
viewdatafile)
Here is the xsd file I was running:
<data-file name="itemdatabasereport.txt"
separator-style="delimited" type-code="UTF-8"
delimiter="	" text-delimiter=""
description="this takes the product data from jeffs file"
has-header="true">
<record name="Product">
<field name="manufacturer_party_id" ignored="false"
type="String"/>
<field name="productId" ignored="false" type="String"/>
<field name="primaryProductCategoryId" expression="true"
default-value="tools" type="String"/>
<field name="productTypeId" expression="true"
default-value="FINISHED_GOOD" type="String"/>
<field name="productName" ignored="false"
type="String"/>
<field name="description" ignored="false"
type="String"/>
<field name="quantityIncluded" expression="true" default-value="1"
type="String"/>
<field name="price" ignored="true" type="String"/>
</record>
</data-file>
What's wrong with my definition file which is causing the data
tool to throw up?
Thanks
James
-----Original Message-----
From: BJ Freeman [mailto:[email protected]]
Sent: November-26-10 1:36 AM
To: [email protected]
Subject: [SPAM]Re: How to import a pre existing product database
I am not clear what you are doing.
the example you show is for an entity DataImportProduct which does
not exist in the OOTB ofbiz take a look at
<https://issues.apache.org/jira/browse/OFBIZ-2534>
https://issues.apache.org/jira/browse/OFBIZ-2534
for tab examples.
also
<https://issues.apache.org/jira/browse/OFBIZ-2518>
https://issues.apache.org/jira/browse/OFBIZ-2518
=========================
BJ Freeman
Strategic Power Office with Supplier Automation<
<http://www.businessesnetwork.com/automation/viewforum.php?f=52>
http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com< <http://www.specialtymarket.com/>
http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
Chat Y! messenger: bjfr33man
James Lawton sent the following on 11/25/2010 6:36 PM:
I am probably doing something stupid here...I have dug into the
link that was posted and it looks simple enough...however the code
snippet is for a fixed width file rather than a comma delimited
file...and my attempts to create the right schema for importing
product are failing...If someone has a CSV file snippet to share
that would be awesome
So heading in a different direction I created the entity data
snippet below:
<!-- this file contains some sample products for testing the data
import routine --> <entity-engine-xml>
<DataImportProduct productId="YG0081ATF"
productTypeId="FINISHED_GOOD" description="1/8 x 1-59/64 CARBIDE
DREAM DRILL W/O COOLANT HOLES (3XD)" price="22.13"
priceCurrencyUomId="USD" supplierPartyId="YG1" purchasePrice=""/>
<DataImportProduct productId="YG0081BTF"
productTypeId="FINISHED_GOOD" description="1/8 x 15/64 CARBIDE
DREAM DRILL W/ COOLANT HOLE" price="105.73"
priceCurrencyUomId="USD"
supplierPartyId="YG1" purchasePrice=""/>
</entity-engine-xml>
However every time I have tried to use the XML IMport tool it just
keeps coming back and saying that the either URL / file or the XML
are not complete
What am I doing wrong?
-----Original Message-----
From: BJ Freeman [mailto:[email protected]]
Sent: November-09-10 6:00 PM
To: [email protected]
Subject: [SPAM]Re: How to import a pre existing product database
all web tools imports into memory so is limited to the amount of
free memory of the sever.
I imports files that are gigs in size so read in the files. I
typical file has 25 columns and takes about 15min.
There is no need to write code to convert files to xml since the
<https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+
Da>
https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+D
a
t
a+
File+Tools
=========================
BJ Freeman
Strategic Power Office with Supplier
Automation<http://www.businessesnetwork.com/automation/viewforum.p
hp?f
=52> Specialtymarket.com< <http://www.specialtymarket.com/>
http://www.specialtymarket.com/>
Systems Integrator-- Glad to Assist
Chat Y! messenger: bjfr33man
James McGill sent the following on 11/9/2010 12:38 PM:
On Tue, Nov 9, 2010 at 9:34 AM, James Lawton<
<mailto:[email protected]>
[email protected]>wrote:
How would I import a pre existing product database? I currently
have it in a CSV and I am looking for a method of getting that
information into the database so that the imported items will
show up in the catalog
OFBiz ETL is best done as XML seed data.
In your scenario I would seriously consider converting the CSV
into XML (a job for Perl), and perhaps doing some input
validation
as part of that conversion, and then loading the XML as seed data.
If you really have a lot of ETL to do (e.g., processing that will
take hours or days to run through the service interface) you
might
be better off going straight to the database. You lose some of
the traceability this way (e.g., logging and event side-effects)
so it isn't recommended. (We ran into problems when loading in
several years worth of historical data, orders and inventory
transactions.)
--
James McGill
Phoenix AZ
