Re: hgserve encoding

[Jérôme Godbout]

After I fix it, eveytime I push with the web server, it work once then I
end up with a
"index 00manifest.i is corrupted"
The repos is dead again and again.
So I can read, push once then it get corrupted?!? If I push with the map
drive locally it work flawlessly.

Any idea what can cause this? this seem to be repeatable if you need
additional information.

[image: bodycad] 
Jerome Godbout
Software Developer
2035 rue du Haut-Bord, Québec, QC, Canada. G1N 4R7
T:  +1 418 527-1388
E: jer...@bodycad.com
www.bodycad.com

The contents of this email message and any attachments are intended solely
for the addressee(s) and may contain confidential and/or privileged
information and may be legally protected from disclosure. If you are not
the intended recipient of this message or their agent, or if this message
has been addressed to you in error, please immediately alert the sender by
reply email and then delete this message and any attachments. If you are
not the intended recipient, you are hereby notified that any use,
dissemination, copying, or storage of this message or its attachments is
strictly prohibited.

Le contenu de ce message et les pièces jointes sont destinés uniquement
pour le(s) destinataire(s) et peuvent contenir des informations
confidentielles et / ou privilégiées qui peuvent être protégées légalement
contre toute divulgation. Si vous n'êtes pas le destinataire de ce message
ou son agent, ou si ce message vous a été adressé par erreur, s’il vous
plaît avertir immédiatement l'expéditeur par courriel de réponse, puis
supprimer ce message et les pièces jointes. Si vous n'êtes pas le
destinataire prévu, vous êtes par la présente informé que toute
utilisation, diffusion, copie, ou stockage de ce message ou de ses pièces
jointes est strictement interdit.

On Sat, Aug 12, 2017 at 9:55 AM, Jérôme Godbout  wrote:

> Finaly found the problems, the 00manifest.i file was corrupted. We
> replaced the main repos .hg with one of the workstation clone and eveything
> is back to normal. Not sure why this got corrupted, but we will keep a
> tight backup on this. I guess the error was early sign of the file
> corruption.
>
> Next step, remove the samba and use NFS!
>
> > On Aug 10, 2017, at 8:34 AM, Yuya Nishihara  wrote:
> >
> > On Wed, 9 Aug 2017 09:09:55 -0400, Jérôme Godbout wrote:
> >> I have some trouble with the hgserve with apache2 with .cgi and my repos
> >> hosting, it mostly work except when it tell me I have corrupted file,
> but
> >> using the repos into local map drive work just fine and verify/recover
> >> doesn't show any problems with it either.
> >>
> >> I known the setup is totally weird, but we try to get ride of many
> Windows
> >> machines (it's a work in progress).
> >>
> >> I have the repos that was originally made under Windows copied to a
> Linux
> >> based machine. The Linux repos file server is sharing it with Samba so
> >> developer can pull/push to it. This work flawlessly.
> >>
> >> I would like to finish the apache2/hg cgi to have all user stop using
> the
> >> map drive and use the web server instead. But right now I have a lot of
> >> weird problems.
> >>
> >> I have a second Linux server that mount the samba share and have the
> >> apache2 with the hgconfig for cgi-bin. I can browse the page, pull from
> it
> >> but when I try to see some history (not all of them), raw or diff for
> some
> >> file it tell me my file are corrupted.
> >
> > Does "hg verify" pass on the samba share? The first thing I would try is
> > to stop using shared filesystem behind hgweb to see if it is a filesystem
> > issue or not.
> >
> >> I think I might have some encoding problems. Since the repos was made
> under
> >> Windows (porbably cp-1251 or wwindows-1251) and I'm using Linux for the
> >> hgweb.
> >
> > Character encoding could be a problem if the repository has files of
> non-ASCII
> > name, and if the underlying filesystem is encoding-aware (e.g. Samba.)
> If the
> > repository is stored in local Unix filesystems (and if clients are
> Windows),
> > filenames are just bytes so cp1251 filenames should just work.
>
>
___
Mercurial mailing list
Mercurial@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial

Re: hgserve encoding

[Jérôme Godbout]

Thanks to clarify the things a bits, I got confuse with the
"Data that Mercurial does locale conversion on includes things like user
names and commit messages...In the case of data taken from files, we should
generally assume that the contents are in the cp1252 charset."
So I wasn't sure what was happening with that. I'm glad it use byte stream
under the hood then. So what is the hgserv.config encoding used for
exactly? is it only web data stream that is mark as encoded into particular
way and it doesn't depend if the repos source? So a repos made under
Windows and Linux would be exactly the same and the OS/FS doesn't matter
for hg repos and access, except the commit metadata (I guess this is true
with what you said).


[image: bodycad] 
Jerome Godbout
Software Developer
2035 rue du Haut-Bord, Québec, QC, Canada. G1N 4R7
T:  +1 418 527-1388
E: jer...@bodycad.com
www.bodycad.com

The contents of this email message and any attachments are intended solely
for the addressee(s) and may contain confidential and/or privileged
information and may be legally protected from disclosure. If you are not
the intended recipient of this message or their agent, or if this message
has been addressed to you in error, please immediately alert the sender by
reply email and then delete this message and any attachments. If you are
not the intended recipient, you are hereby notified that any use,
dissemination, copying, or storage of this message or its attachments is
strictly prohibited.

Le contenu de ce message et les pièces jointes sont destinés uniquement
pour le(s) destinataire(s) et peuvent contenir des informations
confidentielles et / ou privilégiées qui peuvent être protégées légalement
contre toute divulgation. Si vous n'êtes pas le destinataire de ce message
ou son agent, ou si ce message vous a été adressé par erreur, s’il vous
plaît avertir immédiatement l'expéditeur par courriel de réponse, puis
supprimer ce message et les pièces jointes. Si vous n'êtes pas le
destinataire prévu, vous êtes par la présente informé que toute
utilisation, diffusion, copie, ou stockage de ce message ou de ses pièces
jointes est strictement interdit.

On Fri, Aug 11, 2017 at 11:14 AM, Yuya Nishihara  wrote:

> On Thu, 10 Aug 2017 11:58:53 -0400, Jérôme Godbout wrote:
> > I'm not sure if I have find every problems:
> > 1. I have remove the hgweb.config encoding and let him choose.
> > 2. There was also a subrepos that was not copied to the new file server
> > (the personne who did it though that this subrepos is no more used, so no
> > need to copy it! D'oh! still the error is a bit misleading, subrepos not
> > found could have been a funnier error message).
> > Seem to work so far, but I didn't use it much so far. Thanks for the
> > answer, yes we have non ascii file and characters inside some file (Qml
> > file are utf-8, i18n file name might too have non ascii characters).
> >
> > Would be nice if mercurial would be utf-8 all the way. And only the local
> > update would convert to windows encoding from/to local files only (.hg
> into
> > UTF-8 encoding). Would simplify the encoding cross platform mind puzzle.
> I
> > guess that's wishful thinking and would result into other problems, hope
> > one day Windows handle UTF-8 properly all the way around. Maybe I just
> > don't get all the encoding difference with hg and windows/linux, but I
> > still have hard time understand what is in which encoding for
> crossplatform
> > repos, https://www.mercurial-scm.org/wiki/CharacterEncodingOnWindows
> > confuse me more then it help
>
> No idea where you've found the encoding problem. Mercurial treats
> everything
> (except for commit metadata) as bytes. Filenames are bytes and contents are
> also bytes.
>
> Also, UTF-8 can't be a silver bullet. It's reasonable choice only if the
> input
> is a valid unicode (not apply to all Windows encodings.)
>
___
Mercurial mailing list
Mercurial@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial

Re: hgserve encoding

[Yuya Nishihara]

On Thu, 10 Aug 2017 11:58:53 -0400, Jérôme Godbout wrote:
> I'm not sure if I have find every problems:
> 1. I have remove the hgweb.config encoding and let him choose.
> 2. There was also a subrepos that was not copied to the new file server
> (the personne who did it though that this subrepos is no more used, so no
> need to copy it! D'oh! still the error is a bit misleading, subrepos not
> found could have been a funnier error message).
> Seem to work so far, but I didn't use it much so far. Thanks for the
> answer, yes we have non ascii file and characters inside some file (Qml
> file are utf-8, i18n file name might too have non ascii characters).
> 
> Would be nice if mercurial would be utf-8 all the way. And only the local
> update would convert to windows encoding from/to local files only (.hg into
> UTF-8 encoding). Would simplify the encoding cross platform mind puzzle. I
> guess that's wishful thinking and would result into other problems, hope
> one day Windows handle UTF-8 properly all the way around. Maybe I just
> don't get all the encoding difference with hg and windows/linux, but I
> still have hard time understand what is in which encoding for crossplatform
> repos, https://www.mercurial-scm.org/wiki/CharacterEncodingOnWindows
> confuse me more then it help

No idea where you've found the encoding problem. Mercurial treats everything
(except for commit metadata) as bytes. Filenames are bytes and contents are
also bytes.

Also, UTF-8 can't be a silver bullet. It's reasonable choice only if the input
is a valid unicode (not apply to all Windows encodings.)
___
Mercurial mailing list
Mercurial@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial

Re: hgserve encoding

[Yuya Nishihara]

On Wed, 9 Aug 2017 09:09:55 -0400, Jérôme Godbout wrote:
> I have some trouble with the hgserve with apache2 with .cgi and my repos
> hosting, it mostly work except when it tell me I have corrupted file, but
> using the repos into local map drive work just fine and verify/recover
> doesn't show any problems with it either.
> 
> I known the setup is totally weird, but we try to get ride of many Windows
> machines (it's a work in progress).
> 
> I have the repos that was originally made under Windows copied to a Linux
> based machine. The Linux repos file server is sharing it with Samba so
> developer can pull/push to it. This work flawlessly.
> 
> I would like to finish the apache2/hg cgi to have all user stop using the
> map drive and use the web server instead. But right now I have a lot of
> weird problems.
> 
> I have a second Linux server that mount the samba share and have the
> apache2 with the hgconfig for cgi-bin. I can browse the page, pull from it
> but when I try to see some history (not all of them), raw or diff for some
> file it tell me my file are corrupted.

Does "hg verify" pass on the samba share? The first thing I would try is
to stop using shared filesystem behind hgweb to see if it is a filesystem
issue or not.

> I think I might have some encoding problems. Since the repos was made under
> Windows (porbably cp-1251 or wwindows-1251) and I'm using Linux for the
> hgweb.

Character encoding could be a problem if the repository has files of non-ASCII
name, and if the underlying filesystem is encoding-aware (e.g. Samba.) If the
repository is stored in local Unix filesystems (and if clients are Windows),
filenames are just bytes so cp1251 filenames should just work.
___
Mercurial mailing list
Mercurial@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial