Re: [fossil-users] Problem Importing From Git
On Fri, Jul 12, 2013 at 12:24 AM, Aaron W.Hsu arcf...@sacrideo.us wrote:
I have a Git fast-export that comes in at around 322MB in size or
thereabouts. I am running the Windows version of Fossil from 2013-06.
I run the following from within PowerShell and get some funny output:
C:\... git fast-export --all | fossil import --git blah.fossil
]ad fast-import line: [AGS} -fPIC -shared -o $@ $
That ]ad line seems to be incorrect, as at the end of it I get no
blan.fossil file. Can anyone tell me what I am doing wrong here?
I've checked the code again, and found out two things:
1. The error message is mangled because it contains a CR character at
the end (harmless bug).
2. The only situation that, I think, leads to this error is a badly
formatted, badly interpreted or mangled data command. That is,
the number of bytes declared do not match the actual content bytes.
Both of which make me suspicious about an automatic line-ending
conversion going on (LF to CRLF most likely). What I don't really know
is who to blame.
Here are some more tests you can do to help the diagnosis:
- Does the error happen quickly or it takes a while to show up?
- Try the conversion WITHOUT a pipeline. That is:
git fast-export --all my_export
fossil import --git blah.fossil my_export
Note the absence of redirection in the second command.
- Take a look at the my_export file with a text editor or a pager
looking for the same string (${CFLAGS} -fPIC -shared -o $@), then
scroll back until you find a line that starts with data followed
by a number.
Then, extract the content from the data line until the end of the
content (i.e. the whole contents of that particular version of your
Makefile) and calculate its size in bytes. If I'm correct the size
of the Makefile content should approximately match:
content_size = reported_size + linecount_in_content
Once this behaviour is confirmed, the next step is to look out for
guilty parties ;-)
Regards.
--
Isaac Jurado
The noblest pleasure is the joy of understanding
Leonardo da Vinci
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Problem Importing From Git
Dear Isaac: Thanks for helping me through this. In fact, your discussion below triggered the solution. It appears that I have been bitten by a PowerShell feature. Piping data is not the same as in UNIX. When piping, unkown objects are treated as text (at least in this case), and the text itself is converted to a .NET array object before being sent out to the receiving program, at which point, since the receiving program only accepts text (fossil) it will be reconverted back into plain text, but this conversion adds the CRLF line endings instead of the LF line endings that Git uses, thus rendering the data used by Git invalid. There are a few solutions, but as it turns out, the easiest one for working around this particular problem, since this is a one off, is to simply load up cmd as a subshell and then do the processing from there, which works as expected. Fossil works as intended if used from within Cmd for this piping. All in all, I’m glad that this is a fairly straightforward issue. Thanks for the help! Yours truly, Aaron W. Hsu -- Aaron W. Hsu | arcf...@sacrideo.us | http://www.sacrideo.us___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Problem Importing From Git
Dear Isaac:
I’ve managed to figure out how to get some context and the like in Windows.
There are a few lines that have this behavior, so below is the result of
searching for these lines and getting 10 lines of context before and after each
line:
PS C:\... select-string -path exported.txt -pattern 'AGS} -fPIC -shared -o \$@
\$'
-context 10,10
exported.txt:21:cweave $
exported.txt:22:xetex -papersize=letter $*
exported.txt:23:
exported.txt:24:.w.c:
exported.txt:25:ctangle $
exported.txt:26:
exported.txt:27:.c.o:
exported.txt:28:cc ${CFLAGS} -o $@ $
exported.txt:29:
exported.txt:30:.o.so:
exported.txt:31:cc ${CFLAGS} -fPIC -shared -o $@ $
exported.txt:32:blob
exported.txt:33:mark :2
exported.txt:34:data 12798
exported.txt:35:\def\title{HPAPL RUNTIME (VERSION 0.1)}
exported.txt:36:\def\topofcontents{\null\vfill
exported.txt:37: \centerline{\titlefont HPAPL Runtime Library}
exported.txt:38: \vskip 15pt
exported.txt:39: \centerline{(Version 0.1)}
exported.txt:40: \vfill}
exported.txt:41:\def\botofcontents{\vfill
exported.txt:489:cweave $
exported.txt:490:xetex -papersize=letter $*
exported.txt:491:
exported.txt:492:.w.c:
exported.txt:493:ctangle $
exported.txt:494:
exported.txt:495:.c.o:
exported.txt:496:cc ${CFLAGS} -o $@ $
exported.txt:497:
exported.txt:498:.o.so:
exported.txt:499:cc ${CFLAGS} -fPIC -shared -o $@ $
exported.txt:500:commit refs/heads/master
exported.txt:501:mark :5
exported.txt:502:author Aaron W. Hsu arcf...@sacrideo.us 1330040947 -0500
exported.txt:503:committer Aaron W. Hsu arcf...@sacrideo.us 1330040947 -0500
exported.txt:504:data 26
exported.txt:505:Add a better clean target
exported.txt:506:from :3
exported.txt:507:M 100644 :4 runtime/Makefile
exported.txt:508:
exported.txt:509:blob
exported.txt:526:rm -rf *.c *.h *.scn *.idx *.log *.loc *.tex *.dvi *.pdf
*.toc
exported.txt:527:
exported.txt:528:.w.pdf:
exported.txt:529:cweave $
exported.txt:530:xetex -papersize=letter $*
exported.txt:531:
exported.txt:532:.w.c:
exported.txt:533:ctangle $
exported.txt:534:
exported.txt:535:.c.so:
exported.txt:536:cc ${CFLAGS} -fPIC -shared -o $@ $
exported.txt:537:
exported.txt:538:
exported.txt:539:commit refs/heads/master
exported.txt:540:mark :7
exported.txt:541:author Aaron W. Hsu arcf...@sacrideo.us 1330043147 -0500
exported.txt:542:committer Aaron W. Hsu arcf...@sacrideo.us 1330043147 -0500
exported.txt:543:data 28
exported.txt:544:More fixing to the Makefile
exported.txt:545:from :5
exported.txt:546:M 100644 :6 runtime/Makefile
exported.txt:565:rm -rf *.c *.h *.scn *.idx *.log *.loc *.tex *.dvi *.pdf
*.toc *.so
exported.txt:566:
exported.txt:567:.w.pdf:
exported.txt:568:cweave $
exported.txt:569:xetex -papersize=letter $*
exported.txt:570:
exported.txt:571:.w.c:
exported.txt:572:ctangle $
exported.txt:573:
exported.txt:574:.c.so:
exported.txt:575:cc ${CFLAGS} -fPIC -shared -o $@ $
exported.txt:576:
exported.txt:577:
exported.txt:578:blob
exported.txt:579:mark :9
exported.txt:580:data 15138
exported.txt:581:\def\title{HPAPL RUNTIME (VERSION 0.1)}
exported.txt:582:\def\topofcontents{\null\vfill
exported.txt:583: \centerline{\titlefont HPAPL Runtime Library}
exported.txt:584: \vskip 15pt
exported.txt:585: \centerline{(Version 0.1)}
exported.txt:1110:rm -rf *.c *.h *.scn *.idx *.log *.loc *.tex *.dvi
*.pdf *.toc *.so
exported.txt::
exported.txt:1112:.w.pdf:
exported.txt:1113:cweave $
exported.txt:1114:xetex -papersize=letter $*
exported.txt:1115:
exported.txt:1116:.w.c:
exported.txt:1117:ctangle $
exported.txt:1118:
exported.txt:1119:.c.so:
exported.txt:1120:cc ${CFLAGS} -fPIC -shared -o $@ $
exported.txt:1121:
exported.txt:1122:
exported.txt:1123:commit refs/heads/master
exported.txt:1124:mark :12
exported.txt:1125:author Aaron W. Hsu arcf...@sacrideo.us 1330045973 -0500
exported.txt:1126:committer Aaron W. Hsu arcf...@sacrideo.us 1330045973
-0500
exported.txt:1127:data 45
exported.txt:1128:Make sure to have the local path for testing
exported.txt:1129:from :10
exported.txt:1130:M 100644 :11 runtime/Makefile
I hope this helps! I am not sure which line contributes to the failed import.
--
Aaron W. Hsu | arcf...@sacrideo.us | http://www.sacrideo.us___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Problem Importing From Git
El 18/07/2013 20:53, Aaron W.Hsu arcf...@sacrideo.us escribió: [snip] I hope this helps! I am not sure which line contributes to the failed import. I forgot to ask earlier but, do you have non-latin filenames or some textual content. I'm asking because git generates octal escaped sequences (\154) for non-ascii characters in filenames; which Fossil doesn't currently support (patch will be published this weekend). Stay tuned ;-) ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Problem Importing From Git
Dear Isaac: Thanks for the response about non-latin filenames. To my knowledge, I do not have any non-latin filenames, but I do have extensive amounts of non-latin content. However, when it comes to the error with importing in Windows, I think that should not be the problem. The import works fine in a UNIX-like environment, but only fails when I make the attempt on a Windows machine. So, unless the handling of non-latin content is different on Windows and Linux, I would have expected that if it worked on Linux, it would work on Windows. -- Aaron W. Hsu | arcf...@sacrideo.us | http://www.sacrideo.us___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Problem Importing From Git
I should note that in the problem listed below, that this does not happen when I use the Linux version. -- Aaron W. Hsu | arcf...@sacrideo.us | http://www.sacrideo.us From: Aaron W.Hsu Sent: Thursday, July 11, 2013 6:24 PM To: fossil-users@lists.fossil-scm.org I have a Git fast-export that comes in at around 322MB in size or thereabouts. I am running the Windows version of Fossil from 2013-06. I run the following from within PowerShell and get some funny output: C:\... git fast-export --all | fossil import --git blah.fossil ]ad fast-import line: [AGS} -fPIC -shared -o $@ $ That ]ad line seems to be incorrect, as at the end of it I get no blan.fossil file. Can anyone tell me what I am doing wrong here? -- Aaron W. Hsu | arcf...@sacrideo.us | http://www.sacrideo.us___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Problem Importing From Git
El 12/07/2013 00:28, Aaron W.Hsu arcf...@sacrideo.us escribió: I have a Git fast-export that comes in at around 322MB in size or thereabouts. I am running the Windows version of Fossil from 2013-06. I run the following from within PowerShell and get some funny output: C:\... git fast-export --all | fossil import --git blah.fossil ]ad fast-import line: [AGS} -fPIC -shared -o $@ $ Could you locate that line at the export code and provide some context around it? For example, the :mark lines before and after? That should help narrow down the problem. That ]ad line seems to be incorrect, as at the end of it I get no blan.fossil file. Can anyone tell me what I am doing wrong here? I think all the import process is wrapped in a DB transaction, so its all or nothing if an error occurs. Regards. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] problem importing from git
On Jun 29, 2011, at 3:53 AM, John Nowak wrote: $ fossil reconstruct foo.fossil . fossil: SQLITE_ERROR: table blob already exists fossil: table blob already exists Just realized I did this wrong. I'm new to fossil obviously. Still, deconstructing and then reconstructing results in the same error. No error occurs on deconstruction. Still stuck. - jn ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] problem importing from git
On Wed, Jun 29, 2011 at 04:01:35AM -0400, John Nowak wrote: On Jun 29, 2011, at 3:53 AM, John Nowak wrote: $ fossil reconstruct foo.fossil . fossil: SQLITE_ERROR: table blob already exists fossil: table blob already exists Just realized I did this wrong. I'm new to fossil obviously. Still, deconstructing and then reconstructing results in the same error. No error occurs on deconstruction. Still stuck. 'foo.fossil' should not exist before 'reconstruct'. Does it exist? ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] problem importing from git
On Jun 29, 2011, at 4:11 AM, Lluís Batlle i Rossell wrote: On Wed, Jun 29, 2011 at 04:01:35AM -0400, John Nowak wrote: On Jun 29, 2011, at 3:53 AM, John Nowak wrote: $ fossil reconstruct foo.fossil . fossil: SQLITE_ERROR: table blob already exists fossil: table blob already exists Just realized I did this wrong. I'm new to fossil obviously. Still, deconstructing and then reconstructing results in the same error. No error occurs on deconstruction. Still stuck. 'foo.fossil' should not exist before 'reconstruct'. Does it exist? No, I was deconstructing and then reconstructing a new foo2.fossil after I understood how the command works. I can't open that one either though; same not a valid object name error. - jn ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] problem importing from git
On Wed, Jun 29, 2011 at 07:08:52AM -0400, John Nowak wrote: On Jun 29, 2011, at 4:11 AM, Lluís Batlle i Rossell wrote: On Wed, Jun 29, 2011 at 04:01:35AM -0400, John Nowak wrote: On Jun 29, 2011, at 3:53 AM, John Nowak wrote: $ fossil reconstruct foo.fossil . fossil: SQLITE_ERROR: table blob already exists fossil: table blob already exists Just realized I did this wrong. I'm new to fossil obviously. Still, deconstructing and then reconstructing results in the same error. No error occurs on deconstruction. Still stuck. 'foo.fossil' should not exist before 'reconstruct'. Does it exist? No, I was deconstructing and then reconstructing a new foo2.fossil after I understood how the command works. I can't open that one either though; same not a valid object name error. What fossil version? (Not that I know how to fix it, but this may be useful for the code owner who can fix it) ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] problem importing from git
On Wed, Jun 29, 2011 at 3:53 AM, John Nowak j...@johnnowak.com wrote: I am trying to import a very small git repository but I'm running into an error; I've no git branch named trunk if that makes a difference: $ git fast-export --full-tree --all | fossil import --git foo.fossil Rebuilding repository meta-data... 100.0% complete... Vacuuming... ok project-id: 24a1d0ce43a789efe70c1473afb4ff5d303a1620 server-id: 44fbbb7b52d94279f7fed2b2d7ff92b7a4c73ba1 admin-user: jn (password is 57357d) $ fossil open foo.fossil fossil: not a valid object name: trunk Try this: fossil open foo.fossil master I've tried reconstructing; not sure if this is correct: $ fossil reconstruct foo.fossil . fossil: SQLITE_ERROR: table blob already exists fossil: table blob already exists Rebuilding makes no difference. Any ideas? - jn ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users -- D. Richard Hipp d...@sqlite.org ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] problem importing from git
On Jun 29, 2011, at 8:37 AM, Lluís Batlle i Rossell wrote: What fossil version? This is fossil version [0448438c56] 2011-05-28 18:51:22 UTC ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] problem importing from git
On Jun 29, 2011, at 8:47 AM, Richard Hipp wrote: Try this: fossil open foo.fossil master No dice, even though I do have a branch named master as expected: $ fossil open foo.fossil master fossil: not a valid object name: master However, I was able to specify the only other branch and it worked fine. Strange. I think that leaves me stuck with only that branch though; neither master nor trunk shows up in 'fossil ui' (although all of my commits do). I've tried importing other git repositories and they seem to work fine. I'm working with 1.7.4 if that makes a difference. I've noticed that it seems master is remapped to trunk for the repositories that do work... or at least I think so. Is that perhaps not happening properly with this particular repository thus causing the error? - jn ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
