Hi Seb,

Sorry for the trouble, and glad you're at least up and running.

Responses are inline.


On Mon, Aug 7, 2017 at 10:00 PM, Sebastian Unger <sebunge...@gmail.com>
wrote:

> Hi Christian,
>
>>
> Ok, I have some updates. I finally got RB back up and running somewhat. I
> did take the hit and restored from backup. However, at first that failed
> too which allowed me to finally track down the problem to the fact that the
> single (subversion) repository that was causing me this grieve had a
> password configured and this in conjunction with the fact that I did not
> copy the secret key across during a re-deploy caused it (I concluded) to
> fail to decode this password which led to the UTF-8 decoding errors seen in
> the back-trace.
>


Database backups are always great, but don't rely on loaddb/dumpdb for it.
These were built as a way to create a dump for use on other types of
databases, and are not actually supported. To do a database backup, you
should always use your database's own dumping command (like mysqldump).
These will give you a perfect copy of your database, while dumpdb gives you
only an approximation.

The secret key, and site directory in general, should always be considered
part of any site information dump. They go hand-in-hand.



> So once I copied the secret key over, it worked much better, but there are
> still some issues. I'll enumerate them here. I think most of these are not
> related, so if you want me to raise a separate topic/issue for each let me
> know.
>

(Trying to keep the numbers intact, but itemized lists don't seem to
preserve the numbers when replying inline.)



>
>
1. The thing that confused me to no end about the SVN repos is, that the
> username/password settings on the admin page are not used (except to
> produce the aforementioned issues when the secret key is not copied).
> Instead RB uses credentials stored in a .subversion folder inside
> SITEDIR/data. That should probably be fixed.
>

This will depend a bit on your Subversion setup. If the .subversion
directory does not exist, it will be recreated based on the password in the
admin page. However, if the data/ directory is not writable, or if the
password is not correct, it won't be written. If you're using self-signed
certs, though, you will need to re-save the repository entry or copy over
the old .subversion.

It's worth noting that this is all managed by libsvn, and a lot of it is
out of our control.

As mentioned above, you should always use the exact same site directory you
used with the database you're wanting to use.



> 2. One (and only one) of our mercurial repositories produces a crash when
> I select it on the new review request page. All its settings are identical
> to the other repositories served on the same server (via anonymous http).
> The only thing that I can see that is different about this repository is
> that it has only three commits in it.
> If I go to the New Review Request page, select this mercurial repository,
> then in the commit list I get a message "Error 500 Internal Server Error"
> and the stack trace attached as Issue 2 trace is emailed to me.
>

Looks like a bug when attempting to view the last commit in a repository.
Can you file a bug and reference it here? I'll get someone to look at it.



> 3. All subversion repos (again all of the same server, but this time with
> authentication) cause a crash when a particular commit is selected on the
> "New Review Request" page. The list itself works, only when a commit is
> selected, then a pop-up message box says Server Internal Error and I can
> the trace attached as Issue 3 per email.
>

I unfortunately don't have much context to work from on this one, but it
could be a subvertpy bug. Or something a bit odd with some of the data
being returned from the repository that we're using for path building. This
would require more inspection on the server to determine, with logging
information in the product.

Worth noting that while a little more annoying to install on some systems,
PySVN works *much* better than subvertpy.



> 4. I believe that the loaddb command does not properly reset the database.
> The problem code is here
> <https://github.com/reviewboard/reviewboard/blob/release-2.5.15/reviewboard/admin/management/commands/loaddb.py#L53>,
> I believe. It tries to execute ./reviewboard/manage.py as an external
> process. This fails because rb-site must be called from the SITEDIR (or
> else it fails altogether) and thus this path does not point anywhere
> useful. Even if I symlink to reviewboard from the site dir, then manage.py
> does not have the necessary permissions to be executed. If I fail that,
> then it fails because the python path is not set up appropriately. If I fix
> that it turns into issue 6.
>


loaddb does not properly reset the database. As above, it's not really
meant for being a full database backup/restore tool. It's frankly more for
internal stuff during development and some hand-held database transferring
when going from one database type to another.

We will be removing loaddb/dumpdb in 3.0.



> 5. loaddb produces the constraint violations shown in the original post.
> I'm not sure whether these have actually any negative impact, but it would
> probably be better to fix them (or completely remove loaddb).
> This may be a consequence of issue 4. I noticed that interestingly, if I
> loaddb on top of the original DB (i.e. the one I dumped from), I do not get
> the violations.
>

These may have a negative impact, but probably not. But as above, it will
be going away and shouldn't really be used in production.


6. rb-site manage is supposed to be able to run underlying django commands.
> However, running
> *../.venv/bin/rb-site manage "$(pwd)" -- reset reviewboard*
> From my sitedir produces
> *CommandError: App with label reviewboard is missing a models.py module..
> Are you sure your INSTALLED_APPS setting is correct?*
>
>
You can run Django management commands, but "reviewboard" isn't an app with
database models in it. You'd have to specify every single app individually
(reviewboard.reviews, reviewboard.diffviewer, djblets.siteconfig,
django_evolution, etc. -- there's probably about 20 or so of these).
However, you will lose *all* data in those apps, and it's frankly better to
install a brand new site directory at that point with a fresh database.

The "reset" command should really never be used outside of development, as
you run serious risk of outright breaking your database, requiring a full
restore from backup or a full reinstall. The reason is that you can sever
the foreign key constraints from other apps, and impact primary keys in a
way that will mess up all sorts of things. We don't support running it in
production. It's also worth noting that it's not part of Django anymore,
and is nowadays a third-party package that must be manually installed.

Also worth noting that while Django management commands can be run using
rb-site, many Django management commands are either for internal
development purposes or are built with more traditional deployments in
mind. Review Board has a more custom environment, in order for us to handle
things like deployments and upgrades, and won't *always* work with all
commands.



> I'd say 2+3 are critical. 1 is important as it caused us repeated
> confusion when that hidden .subversion directory was not copied and then
> the SVN repos all stopped working (even though they seemed to have correct
> credentials configured in the admin api).
> 4+5 are loaddb related and only important if that interface isn't
> completely dropped.
> 6 is important because it makes other trouble worse by not being able to
> use the underlying Django commands to troubleshoot anything.
>

2 and 3 definitely need to be fixed, and I'd love to get bug reports with
information on those. We'd have to help figure out how to gather more
detailed info on 3, but 2 should be pretty easily fixable.

I'm actually going to be on vacation for the next week (starting in 30
minutes), so I won't be around to follow up on this until I'm back. If
there are any emergency issues that come up, others on my team will take a
look.

Thanks!

Christian



>
> Cheers,
> Seb
>
> --
> Supercharge your Review Board with Power Pack:
> https://www.reviewboard.org/powerpack/
> Want us to host Review Board for you? Check out RBCommons:
> https://rbcommons.com/
> Happy user? Let us know! https://www.reviewboard.org/users/
> ---
> You received this message because you are subscribed to the Google Groups
> "reviewboard" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to reviewboard+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Christian Hammond
President/CEO of Beanbag <https://www.beanbaginc.com/>
Makers of Review Board <https://www.reviewboard.org/>

-- 
Supercharge your Review Board with Power Pack: 
https://www.reviewboard.org/powerpack/
Want us to host Review Board for you? Check out RBCommons: 
https://rbcommons.com/
Happy user? Let us know! https://www.reviewboard.org/users/
--- 
You received this message because you are subscribed to the Google Groups 
"reviewboard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to reviewboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to