Re: [petsc-dev] Apply for Google Summer of Code 2023?

2023-02-22 Thread Karl Rupp
this as input for a better application next year :-) Best regards, Karli On 2/7/23 18:37, Karl Rupp wrote: Dear all, thanks for all the input and help. Our application has been submitted, let's keep our fingers crossed. Also, this is a friendly reminder to fill out the details on the GSoC-topics

Re: [petsc-dev] Apply for Google Summer of Code 2023?

2023-02-07 Thread Karl Rupp
are properly communicated. :-) Thanks and best regards, Karli On 2/6/23 20:24, Karl Rupp wrote: Hello all, thanks for proposing projects. I've created the suggestions so far as 'issues' in the issue tracker on Gitlab, prefixed by 'GSoC:'. Please add a better description to your suggestions so

Re: [petsc-dev] Apply for Google Summer of Code 2023?

2023-02-06 Thread Karl Rupp
ust async * Differentiable programming using Enzyme with PETSc I like all those.   Matt Karl Rupp mailto:r...@iue.tuwien.ac.at>> writes: > Dear PETSc developers, > > in order to attract students to PETSc development, I'm thinking about a > PETSc ap

[petsc-dev] Apply for Google Summer of Code 2023?

2023-02-03 Thread Karl Rupp
Dear PETSc developers, in order to attract students to PETSc development, I'm thinking about a PETSc application for Google Summer of Code (GSoC) 2023: https://summerofcode.withgoogle.com/programs/2023 The org application deadline is February 7, i.e. in 4 days. This application is - roughly

Re: [petsc-dev] empty space on left side of website pages

2021-04-27 Thread Karl Rupp
Hi, such adjustments should not need a direct modification of the theme. One can just override the CSS settings in custom CSS files instead: https://docs.readthedocs.io/en/stable/guides/adding-custom-css.html The benefit of such an approach is that all future updates of the theme will

Re: [petsc-dev] PETSc issue I cannot post combine WaitForCUDA(); inside PetscLogGpuTimeEnd();

2020-08-28 Thread Karl Rupp
 Since we cannot post issues (reported here https://forum.gitlab.com/t/creating-new-issue-gives-cannot-create-issue-getting-whoops-something-went-wrong-on-our-end/41966?u=bsmith) here is my issue so I don't forget it.  I think err  = WaitForCUDA();CHKERRCUDA(err); ierr =

Re: [petsc-dev] PETSc issue I cannot post combine WaitForCUDA(); inside PetscLogGpuTimeEnd();

2020-08-28 Thread Karl Rupp
Hi,   Since we cannot post issues (reported here https://forum.gitlab.com/t/creating-new-issue-gives-cannot-create-issue-getting-whoops-something-went-wrong-on-our-end/41966?u=bsmith) here is my issue so I don't forget it.   I think  err  = WaitForCUDA();CHKERRCUDA(err);  ierr =

Re: [petsc-dev] https://developer.nvidia.com/nccl

2020-06-16 Thread Karl Rupp
From a practical standpoint it seems to me that NCCL is an offering to a community that isn't used to MPI. It's categorized as 'Deep Learning Software' on the NVIDIA page ;-) The section 'NCCL and MPI' has some interesting bits:

Re: [petsc-dev] Valgrind MPI-Related Errors

2020-06-01 Thread Karl Rupp
Hi Jacob, the recommendation in the past was to use MPICH as it is (was?) valgrind-clean. Which MPI do you use? OpenMPI used to have these kinds of issues. (My information might be outdated) Best regards, Karli On 6/2/20 2:43 AM, Jacob Faibussowitsch wrote: Hello All, TL;DR: valgrind

Re: [petsc-dev] snes_tutorials-ex19_cuda_1

2020-04-01 Thread Karl Rupp
The fluctuations in this example have been fixed a few months ago; the issue was the use of multiple streams instead of a single one. Maybe additional CUDA streams have been reintroduced recently? Best regards, Karli On 4/2/20 5:02 AM, Junchao Zhang wrote: I could not reproduce it locally.

[petsc-dev] No PETSc User Meeting in 2020

2020-03-14 Thread Karl Rupp
Dear PETSc developers and PETSc users, due to the recent Covid-19 outbreak in Europe there will not be a PETSc User Meeting this year. We are looking into alternatives for keeping in touch with our user base, e.g. via webinars. Suggestions welcome :-) Thanks and best regards, Karl

Re: [petsc-dev] PETSc meeting not on web page ...

2020-02-24 Thread Karl Rupp
CUDA code). Thanks for the discussion at PP20. On Sat, Feb 22, 2020 at 11:16 PM Karl Rupp <mailto:r...@iue.tuwien.ac.at>> wrote: Hi Mark, we are just finalizing the last few details (in particular: which registration system to use) before sending out the anno

Re: [petsc-dev] Proper matrix size to choose when evaluating MatMult?

2020-02-22 Thread Karl Rupp
Hi Junchao, I want to evaluate MatMult on GPU.  I took a 2M x 2M matrix and ran with 6 mpi ranks and 6 GPUs.  It took about 0.9 seconds. How many nonzeros per row? With 0.9 seconds you should either have many runs of MatMult, or a fairly dense matrix; or a really slow MatMult kernel ;-)

Re: [petsc-dev] PETSc meeting not on web page ...

2020-02-22 Thread Karl Rupp
Hi Mark, we are just finalizing the last few details (in particular: which registration system to use) before sending out the announcements and putting the link on the main webpage. Just a matter of a few days. :-) Best regards, Karli On 2/22/20 9:11 PM, Mark Adams wrote: Maybe the

Re: [petsc-dev] First call to cudaMalloc or cudaFree is very slow on summit

2020-02-13 Thread Karl Rupp
Hi Hong, have you tried running the code through gprof and look at the output (e.g. with kcachegrind)? (apologies if this has been suggested already) Best regards, Karli On 2/12/20 7:29 PM, Zhang, Hong via petsc-dev wrote: On Feb 12, 2020, at 5:11 PM, Smith, Barry F. wrote: ldd

Re: [petsc-dev] Feed back on report on performance of vector operations on Summit requested

2019-10-09 Thread Karl Rupp via petsc-dev
Hi, Table 2 reports negative latencies. This doesn't look right to me ;-) If it's the outcome of a parameter fit to the performance model, then use a parameter name (e.g. alpha) instead of the term 'latency'. Figure 11 has a very narrow range in the y-coordinate and thus exaggerates the

Re: [petsc-dev] Why no SpGEMM support in AIJCUSPARSE and AIJVIENNACL?

2019-10-04 Thread Karl Rupp via petsc-dev
to provide a good playground on time for the Summit hackathon. In the meantime you can try the matrix market reader of nsparse directly and see what you get, especially compared to cuSPARSE and MKL. Best regards, Karli Karl Rupp via petsc-dev writes: Hi Richard, CPU spGEMM is about

Re: [petsc-dev] Why no SpGEMM support in AIJCUSPARSE and AIJVIENNACL?

2019-10-03 Thread Karl Rupp via petsc-dev
and compare against the other packages. In the end it doesn't matter which package provides the best performance; we just want to leverage it :-) Best regards, Karli Karl Rupp via petsc-dev writes: Hi Richard, CPU spGEMM is about twice as fast even on the GPU-friendly case of a single

Re: [petsc-dev] Why no SpGEMM support in AIJCUSPARSE and AIJVIENNACL?

2019-10-02 Thread Karl Rupp via petsc-dev
Hi Richard, CPU spGEMM is about twice as fast even on the GPU-friendly case of a single rank: http://viennacl.sourceforge.net/viennacl-benchmarks-spmm.html I agree that it would be good to have a GPU-MatMatMult for the sake of experiments. Under these performance constraints it's not top

Re: [petsc-dev] Should v->valid_GPU_array be a bitmask?

2019-10-02 Thread Karl Rupp via petsc-dev
Hi Junchao, I recall that Jed already suggested to make this a bitmask ~7 years ago ;-) On the other hand: If we touch valid_GPU_array, then we should also use a better name or refactor completely. Code like (V->valid_GPU_array & PETSC_OFFLOAD_GPU) simply isn't intuitive (nor does it make

Re: [petsc-dev] error with karlrupp/fix-cuda-streams

2019-09-27 Thread Karl Rupp via petsc-dev
Hi Mark, OK, so now the problem has shifted somewhat in that it now manifests itself on small cases. In earlier investigation I was drawn to MatTranspose but had a hard time pinning it down. The bug seems more stable now or you probably fixed what looks like all the other bugs. I added

Re: [petsc-dev] error with karlrupp/fix-cuda-streams

2019-09-25 Thread Karl Rupp via petsc-dev
On Wed, Sep 25, 2019 at 5:26 AM Karl Rupp via petsc-dev mailto:petsc-dev@mcs.anl.gov>> wrote: On 9/25/19 11:12 AM, Mark Adams via petsc-dev wrote: > I am using karlrupp/fix-cuda-streams, merged with master, and I get this > error: > > Could not execu

Re: [petsc-dev] error with karlrupp/fix-cuda-streams

2019-09-25 Thread Karl Rupp via petsc-dev
On 9/25/19 11:12 AM, Mark Adams via petsc-dev wrote: I am using karlrupp/fix-cuda-streams, merged with master, and I get this error: Could not execute "['jsrun -g\\ 1 -c\\ 1 -a\\ 1 --oversubscribe -n 1 printenv']": Error, invalid argument:  1 My branch mark/fix-cuda-with-gamg-pintocpu 

Re: [petsc-dev] MatMult on Summit

2019-09-24 Thread Karl Rupp via petsc-dev
ichard On 9/23/19 8:09 PM, Karl Rupp wrote: Hi, `git grep cudaStreamCreate` reports that vectors, matrices and scatters create their own streams. This will almost inevitably create races (there is no synchronization mechanism implemented), unless one calls WaitForGPU()

Re: [petsc-dev] MatMult on Summit

2019-09-23 Thread Karl Rupp via petsc-dev
Hi, `git grep cudaStreamCreate` reports that vectors, matrices and scatters create their own streams. This will almost inevitably create races (there is no synchronization mechanism implemented), unless one calls WaitForGPU() after each operation. Some of the non-deterministic tests can

Re: [petsc-dev] MatMult on Summit

2019-09-21 Thread Karl Rupp via petsc-dev
On 9/22/19 6:15 AM, Jed Brown wrote: Karl Rupp via petsc-dev writes: Hi Junchao, thanks, these numbers are interesting. Do you have an easy way to evaluate the benefits of a CUDA-aware MPI vs. a non-CUDA-aware MPI that still keeps the benefits of your packing/unpacking routines? I'd

Re: [petsc-dev] MatMult on Summit

2019-09-21 Thread Karl Rupp via petsc-dev
Hi Junchao, thanks, these numbers are interesting. Do you have an easy way to evaluate the benefits of a CUDA-aware MPI vs. a non-CUDA-aware MPI that still keeps the benefits of your packing/unpacking routines? I'd like to get a feeling of where the performance gains come from. Is it due

Re: [petsc-dev] hypre and CUDA

2019-08-15 Thread Karl Rupp via petsc-dev
Hi, one way to test is to run a sequential example through nv-prof: $> nvprof ./ex56 ... https://devblogs.nvidia.com/cuda-pro-tip-nvprof-your-handy-universal-gpu-profiler/ If it uses the GPU, then you will get some information on the GPU kernels called. If it doesn't use the GPU, the list

[petsc-dev] Schedule for next release

2019-08-14 Thread Karl Rupp via petsc-dev
Hi all, let me propose the following schedule for the next release: * until Sunday, September 15: New pull requests are considered for the upcoming release. * from Monday, September 16, to Sunday, September 22: Fixing and merging of open pull requests received by September 15. Extended

Re: [petsc-dev] Is master broken?

2019-08-13 Thread Karl Rupp via petsc-dev
non-collective VecSet 12042c4bfa removing ViennaCL fix to GAMG 3c46958f6d fix bug with empty processor 8bcb2d50b7 fixed MPI lock from call to collective method 54cfeb1831 added missing settypes 9508265e8e adding support for MatTranspose e5a6000419 adding fix for ViennaCL in MG On Mon, 12 Aug

Re: [petsc-dev] Is master broken?

2019-08-12 Thread Karl Rupp via petsc-dev
? (on vacation this week) I will verify that master is all fixed if you get all my stuff integrated when I get back to work in a week. Thanks, Mark On Sat, Aug 3, 2019 at 10:50 AM Karl Rupp <mailto:r...@iue.tuwien.ac.at>> wrote: If you ignore the initial ViennaCL-related commits and chec

Re: [petsc-dev] Is master broken?

2019-08-03 Thread Karl Rupp via petsc-dev
fixed if you get all my stuff integrated when I get back to work in a week. Thanks, Mark On Sat, Aug 3, 2019 at 10:50 AM Karl Rupp <mailto:r...@iue.tuwien.ac.at>> wrote: If you ignore the initial ViennaCL-related commits and check against current master (that just received cher

Re: [petsc-dev] Is master broken?

2019-08-03 Thread Karl Rupp via petsc-dev
If you ignore the initial ViennaCL-related commits and check against current master (that just received cherry-picked updates from your PR), then there are really only a few commits left that are not yet integrated. (I'll extract two more PRs on Monday, so master will soon have your fixes

Re: [petsc-dev] Is master broken?

2019-08-02 Thread Karl Rupp via petsc-dev
Barry's branch. Or will that not work. On Fri, Aug 2, 2019 at 12:03 PM Karl Rupp <mailto:r...@iue.tuwien.ac.at>> wrote: FYI: The two branches are currently testing in `next-tmp` and are likely to be merged to master in ~5 hours. Best regards, Karli On 8/2/1

Re: [petsc-dev] Is master broken?

2019-08-02 Thread Karl Rupp via petsc-dev
FYI: The two branches are currently testing in `next-tmp` and are likely to be merged to master in ~5 hours. Best regards, Karli On 8/2/19 4:53 PM, Smith, Barry F. via petsc-dev wrote: Yes, these are bugs in Stefano's work that got into master because we didn't have comprehensive

Re: [petsc-dev] MatPinToCPU

2019-07-28 Thread Karl Rupp via petsc-dev
Hi Mark, feel free to submit a fresh pull request now. I looked at your latest commit in the repository in order to cherry-pick it, but it looked like it had a few other bits in it as well. Best regards, Karli On 7/28/19 6:27 PM, Mark Adams via petsc-dev wrote: This is looking good. I'm

Re: [petsc-dev] valid_GPU_matrix in interface code

2019-07-23 Thread Karl Rupp via petsc-dev
Hi Stefano, I have just noticed we have different occurrences of the valid_GPU_matrix flag in src/mat/interface and src/mat/utils I think that how they are used now is wrong, as they assume that all those operations can only be executed on the CPU, irrespective of the specific type. Is there

Re: [petsc-dev] Bitbucket is doomed

2019-05-29 Thread Karl Rupp via petsc-dev
That's just a manifestation of Satish merging really well today ;-) Best regards, Karli On 5/30/19 1:11 AM, Smith, Barry F. via petsc-dev wrote:    I just got this same merged message sent to me three times.    In recent days I've received several sent to me twice.    It's not like we

Re: [petsc-dev] alternatives to alt files

2019-05-02 Thread Karl Rupp via petsc-dev
Using alt files for testing is painful. Whenever you add, for example, a new variable to be output in a viewer it changes the output files and you need to regenerate the alt files for all the test configurations. Even though the run behavior of the code hasn't changed. I'm

Re: [petsc-dev] alternatives to alt files

2019-05-02 Thread Karl Rupp via petsc-dev
Hi, Scott and PETSc folks, Using alt files for testing is painful. Whenever you add, for example, a new variable to be output in a viewer it changes the output files and you need to regenerate the alt files for all the test configurations. Even though the run behavior of the code

Re: [petsc-dev] [petsc-checkbuilds] PETSc blame digest (next) 2019-04-26

2019-04-26 Thread Karl Rupp via petsc-dev
Hi, I fixed this warning after merge. Best regards, Karli On 4/26/19 2:28 PM, PETSc checkBuilds via petsc-checkbuilds wrote: Dear PETSc developer, This email contains listings of contributions attributed to you by `git blame` that caused compiler errors or warnings in PETSc automated

Re: [petsc-dev] Thoughts on pushing current CI infrastructure to the next level

2019-04-25 Thread Karl Rupp via petsc-dev
On 4/25/19 6:53 PM, Jed Brown wrote: Karl Rupp via petsc-dev writes: With some effort we can certainly address 1.) and to some extent 3.), probably 4.) as well, but I don't know how to solve 2.) and 5.) with Jenkins. Given that a significant effort is required for 1.), 3.) and 4.) anyway

[petsc-dev] Thoughts on pushing current CI infrastructure to the next level

2019-04-25 Thread Karl Rupp via petsc-dev
Dear PETSc developers, the current Jenkins server went live last summer. Since then, the stability of master and next has indeed improved. Who would have thought three years ago that `next` is almost as stable as `master`? However, over the weeks and months some weaknesses of our current

Re: [petsc-dev] https://www.dursi.ca/post/hpc-is-dying-and-mpi-is-killing-it.html

2019-03-18 Thread Karl Rupp via petsc-dev
Hi Matt, (...) His slides have more, " PETSc is a widely used library for large sparse iterative solves.         Excellent and comprehensive library of solvers         It is the basis of a significant number of home-made simulation codes         It is notoriously 

Re: [petsc-dev] Is there a good reason that BuildSystem's cuda.py requires GNU compilers?

2019-03-14 Thread Karl Rupp via petsc-dev
Hi Richard, the check for the GNU compilers is mostly a historic relic. We haven't done any systematic tests with other compilers, so that test has just remained in place. It would certainly be good if you could update the check to also work well with the default environment on Summit.

Re: [petsc-dev] PETSc release by March 29, 2019

2019-03-05 Thread Karl Rupp via petsc-dev
arch 22 [i.e anything that would be acceptable in our maint work-flow shouldn't be frozen] - And we should be able to drop troublesome PRs if they are blocking the release. full ack :-) Best regards, Karli Satish On Tue, 5 Mar 2019, Karl Rupp via petsc-dev wrote: Dear PETSc develo

Re: [petsc-dev] PETSc release by March 29, 2019

2019-03-05 Thread Karl Rupp via petsc-dev
Dear PETSc developers, let me suggest Friday, March 22, as the cut-off-date for new Pull Requests for the upcoming release. This allows for 7 days to iron out any remaining glitches. (It only took us a few days to release after the cut-off date last September, so this should be fine) Also,

Re: [petsc-dev] [SPAM?] Re: Checkpoint-restart with DMPlex objects

2018-12-18 Thread Karl Rupp via petsc-dev
I have not quickly found how is that "VTK ordering" defined but I hopefully it's a well-defined unambiguous cell-local numbering. I will try to find it out soon and get back to you. Hope this helps: https://www.vtk.org/wp-content/uploads/2015/04/file-formats.pdf (page 9) Best regards,

Re: [petsc-dev] Error running on Titan with GPUs & GNU

2018-10-31 Thread Karl Rupp via petsc-dev
gt; wrote: On Wed, Oct 31, 2018 at 5:05 AM Karl Rupp mailto:r...@iue.tuwien.ac.at>> wrote: Hi Mark, please comment or remove lines 83 and 84 in   config/BuildSystem/config/packages/cuda.py Is there a compiler newer than GCC 4.3 available?

Re: [petsc-dev] Error running on Titan with GPUs & GNU

2018-10-31 Thread Karl Rupp via petsc-dev
Hi Mark, please comment or remove lines 83 and 84 in config/BuildSystem/config/packages/cuda.py Is there a compiler newer than GCC 4.3 available? Best regards, Karli On 10/31/18 8:15 AM, Mark Adams via petsc-dev wrote: After loading a cuda module ... On Wed, Oct 31, 2018 at 2:58 AM Mark

Re: [petsc-dev] Code of Conduct [ACTION REQUIRED]

2018-10-26 Thread Karl Rupp
Dear PETSc folks, after broad approval, the pull request for the Code of Conduct has been merged to master and is now active. Best regards, Karli On 10/23/18 12:52 PM, Karl Rupp wrote: Dear PETSc folks, I ask all members of the PETSc team to review the following proposal for adopting

[petsc-dev] Code of Conduct [ACTION REQUIRED]

2018-10-23 Thread Karl Rupp
Dear PETSc folks, I ask all members of the PETSc team to review the following proposal for adopting a code of conduct: https://bitbucket.org/petsc/petsc/pull-requests/1196/code-of-conduct-adopt-contributor-covenant/diff If you have questions, concerns, etc., please reply to this email

Re: [petsc-dev] PETSc 3.10: September 4, 2018, is the cutoff-date for new features

2018-09-13 Thread Karl Rupp
2018, Karl Rupp wrote: Dear PETSc developers, please open any outstanding pull requests for the upcoming PETSc 3.10 release in the next few hours. After that, please do not merge anything to `next` or `master` unless it is integration work for existing open PRs. You can open up new pull requ

Re: [petsc-dev] PETSc 3.10: September 4, 2018, is the cutoff-date for new features

2018-09-04 Thread Karl Rupp
, Karli On 08/30/2018 06:09 PM, Karl Rupp wrote: Dear PETSc developers, this is a gentle reminder for the cutoff-date on September 4. Best regards, Karli On 07/27/2018 02:41 AM, Karl Rupp wrote: Dear PETSc developers, in order to ensure a PETSc 3.10 release no later than by the end

Re: [petsc-dev] PETSc 3.10: September 4, 2018, is the cutoff-date for new features

2018-08-30 Thread Karl Rupp
Dear PETSc developers, this is a gentle reminder for the cutoff-date on September 4. Best regards, Karli On 07/27/2018 02:41 AM, Karl Rupp wrote: Dear PETSc developers, in order to ensure a PETSc 3.10 release no later than by the end of September (possibly earlier), we agreed on September

[petsc-dev] PETSc 3.10: September 4, 2018, is the cutoff-date for new features

2018-07-26 Thread Karl Rupp
Dear PETSc developers, in order to ensure a PETSc 3.10 release no later than by the end of September (possibly earlier), we agreed on September 4, 2018, as the cut-off date for new features. Please make sure that a pull request has been opened on Bitbucket by this time. This is the preferred

Re: [petsc-dev] Bad scaling of GAMG in FieldSplit

2018-07-26 Thread Karl Rupp
Hi Pierre, I’m using GAMG on a shifted Laplacian with these options: -st_fieldsplit_pressure_ksp_type preonly -st_fieldsplit_pressure_pc_composite_type additive -st_fieldsplit_pressure_pc_type composite -st_fieldsplit_pressure_sub_0_ksp_pc_type jacobi -st_fieldsplit_pressure_sub_0_pc_type ksp

Re: [petsc-dev] PETSc goes Jenkins

2018-07-20 Thread Karl Rupp
assed", even though failed tests are reported (timeouts). 2018-07-20 3:35 GMT+02:00 Karl Rupp <mailto:r...@iue.tuwien.ac.at>>: Hi all, we now have a first step towards full continuous integration via Jenkins completed. Thus, every new pull request that is (re-)based on a

[petsc-dev] PETSc goes Jenkins

2018-07-19 Thread Karl Rupp
Hi all, we now have a first step towards full continuous integration via Jenkins completed. Thus, every new pull request that is (re-)based on a commit in master not older than today will be automatically tested with a subset of common tests that are intended to expose the most frequent

Re: [petsc-dev] Could we replace SNESTestJacobian() in maint with that in master?

2018-07-18 Thread Karl Rupp
Hi Fande, It looks like SNESTestJacobian() in master is more reliable than that in maint.  Especially, the petsc options name is changed. For example, from "-snes_test_jacobian_display" to "-snes_test_jacobian_view". There are some MOOSE tests that fail with the maint version of

Re: [petsc-dev] PetscSF and/or VecScatter with device pointers

2018-07-14 Thread Karl Rupp
Hi, we're starting to explore (with Andreas cc'd) residual assembly on GPUs. The question naturally arises: how to do GlobalToLocal and LocalToGlobal. I have: A PetscSF describing the communication pattern. A Vec holding the data to communicate. This will have an up-to-date device

Re: [petsc-dev] GAMG error with MKL

2018-07-07 Thread Karl Rupp
Hi all, (...)Since it looks like MPI endpoints are going to be a long time (or possibly forever) in coming, I think we need (a) stopgap plan(s) to support this crappy MPI + OpenMP model in the meantime. One possible approach is to do what Mark is trying with to do with MKL: Use a third

Re: [petsc-dev] building with MKL

2018-07-01 Thread Karl Rupp
Hi Mark, have a look at config/examples/arch-linux-knl.py, which contains on line 20: '--with-blaslapack-dir='+os.environ['MKLROOT'], It's important that you specify the BLAS library *and* the MKL include directory (either via --with-blaslapack-dir or via a pair of --with-blaslapack-include

Re: [petsc-dev] HDF5 download error

2018-06-20 Thread Karl Rupp
Hi Mark, the FTP server at MCS is down today. It should come back up later today. Best regards, Karli On 06/20/2018 01:17 PM, Mark Adams wrote: This looks like it is a problem with NERSC, this does not work: 04:14 cori04 maint= ~/petsc_install/petsc$ ping ftp.mcs.anl.gov

Re: [petsc-dev] Tiny pull requests via Bitbucket web interface

2018-06-06 Thread Karl Rupp
Hi all, yes, I support Patrick's idea of actively encouraging such simple pull requests. Particularly when it comes to documentation, it would be very handy to also add a link to the manual pages on the top right. For example,

Re: [petsc-dev] Figures 20, 21 in Chapter 13.1 of PETSc manual are out of date

2018-04-18 Thread Karl Rupp
Hi Junchao, 1) The manual says the example is src/ksp/ksp/examples/ex10.c, but it actually links to src/ksp/ksp/examples/tutorial/ex10.c. This is a minor issue. 2) One could not use the same command line options (-f0 medium -f1 arco6) as shown in the figures. There are no such matrices so one

Re: [petsc-dev] upcoming release and testing [v2]

2018-04-06 Thread Karl Rupp
own kernels. As far as I know, CUBLAS and CUSPARSE initialization is fairly slow on the M2090. Best regards, Karli On 04/06/2018 09:13 PM, Karl Rupp wrote: Hi, The CUDA tests are hanging/timing-out more often now. For eg: http://ftp.mcs.anl.gov/pub/petsc/nightlylogs/archive/2018/04/06

Re: [petsc-dev] upcoming release and testing [v2]

2018-04-06 Thread Karl Rupp
Hi, The CUDA tests are hanging/timing-out more often now. For eg: http://ftp.mcs.anl.gov/pub/petsc/nightlylogs/archive/2018/04/06/examples_next_arch-cuda-double_es.log And I did see some build where they didn't get killed due to timeout. For eg:

Re: [petsc-dev] upcoming release and testing

2018-04-05 Thread Karl Rupp
Hi Satish, FYI: I added a mention of GPU backends available in the release and fixed missing ul-tags in src/docs/website/documentation/changes/39.html in your balay/release-3.9 branch. Best regards, Karli On 04/02/2018 08:18 PM, Satish Balay wrote: All, It would be good if

Re: [petsc-dev] upcoming release and testing

2018-04-02 Thread Karl Rupp
Hi Satish, CUDA and ViennaCL can be enabled for the first time in this release. Best regards, Karli On 04/02/2018 08:15 PM, Satish Balay wrote: Karl, Are we disabling CUDA usage for this release aswell? Thanks, Satish

Re: [petsc-dev] upcoming release and testing

2018-04-01 Thread Karl Rupp
Hi Satish, I'll try to send follow up emails on master brakages. Karl, http://ftp.mcs.anl.gov/pub/petsc/nightlylogs/archive/2018/03/31/examples_master_arch-c-exodus-dbg-builder_es.log not ok ksp_ksp_tests-ex43_1 # terminate called after throwing an instance of

Re: [petsc-dev] PETSc blame digest (next) 2018-03-28

2018-03-28 Thread Karl Rupp
Hi, fixes for these are now in next. Best regards, Karli On 03/28/2018 04:00 PM, PETSc checkBuilds wrote: Dear PETSc developer, This email contains listings of contributions attributed to you by `git blame` that caused compiler errors or warnings in PETSc automated testing. Follow the

Re: [petsc-dev] Target next PETSc/TAO release for March 30

2018-03-27 Thread Karl Rupp
:31 GMT+02:00 Karl Rupp <r...@iue.tuwien.ac.at <mailto:r...@iue.tuwien.ac.at>>: Hi Stefano, Next is quite messy at the moment. How are we going to handle merging to master? I have a couple of branches that are sitting in next for a while, and tha

Re: [petsc-dev] Target next PETSc/TAO release for March 30

2018-03-26 Thread Karl Rupp
Hi Stefano, Next is quite messy at the moment. How are we going to handle merging to master? I have a couple of branches that are sitting in next for a while, and that can be merged safely : stefano_zampini/fix-matis-dmda-l2g and stefano_zampini/add-pod-test I merged

Re: [petsc-dev] [petsc-maint] Installing with CUDA on a cluster

2018-03-12 Thread Karl Rupp
since driver updates are much easier than full OS upgrades. Best regards, Karli And will plan on using M2090 testbed [with either cuda-7.5 or cuda-8 -arch=sm_20] for forseeable future. Satish On Mon, 12 Mar 2018, Karl Rupp wrote: Hi Satish, thanks for the pull request. I approve the chang

Re: [petsc-dev] [petsc-maint] Installing with CUDA on a cluster

2018-03-12 Thread Karl Rupp
defaults? Then we can probably eliminate all this complicated code. [If cuda-7.5 and higer do this properly - we could use that as the minimum supported version?] Satish On Sat, 10 Mar 2018, Karl Rupp wrote: Hi all, a couple of notes here, particularly for Manuel: * CUSP is repeatedly

Re: [petsc-dev] plans for preconditioners for SeqSELL

2018-03-06 Thread Karl Rupp
ons and preconditioners. Best regards, Karli --Richard On 3/4/18 2:58 AM, Karl Rupp wrote: Hi all, I'm getting increasingly concerned about SELL not being a subclass of AIJ. As such, we have to deal with all these fallback operations now, whereas as a subclass of AIJ we could just selectively m

Re: [petsc-dev] Handling pull requests in a better way

2018-03-06 Thread Karl Rupp
Hi Richard, I'm a bit late to the discussion, but I want to point out one of the issues I've encountered with pull requests: Often a pull request is submitted with multiple reviewers listed, and it's sometimes not clear how many of the reviewers need to look at it. I've spent some time

Re: [petsc-dev] Handling pull requests in a better way

2018-03-04 Thread Karl Rupp
Hi, since nobody explicitly objected and since nobody volunteered for the PR integrator role, I'll take over this role for the next month or two. Let's evaluate the process then. Best regards, Karli On 03/01/2018 12:33 PM, Karl Rupp wrote: Dear PETSc folks, I think we can do a better job

Re: [petsc-dev] plans for preconditioners for SeqSELL

2018-03-04 Thread Karl Rupp
r solvers etc? > > > > > > 2018-02-12 18:06 GMT+03:00 Stefano Zampini <stefano.zamp...@gmail.com <mailto:stefano.zamp...@gmail.com>>: > > > > > > 2018-02-12 17:36 GMT+03:00 Jed Brown <j...@jedbrown.org

Re: [petsc-dev] Handling pull requests in a better way

2018-03-01 Thread Karl Rupp
, Karli On Mar 1, 2018, at 5:33 AM, Karl Rupp <r...@iue.tuwien.ac.at> wrote: Dear PETSc folks, I think we can do a better job when it comes to handling pull requests (PRs). We have several PRs piling up, which after some time (imho) get merged relatively carelessly instead of r

[petsc-dev] Handling pull requests in a better way

2018-03-01 Thread Karl Rupp
Dear PETSc folks, I think we can do a better job when it comes to handling pull requests (PRs). We have several PRs piling up, which after some time (imho) get merged relatively carelessly instead of reaping the full benefits of a thorough review. In order to improve the integration of pull

[petsc-dev] Release schedule?

2018-02-26 Thread Karl Rupp
Hi, what is the current release schedule? Are we supposed to release a new PETSc version in March (maybe ECP-related)? I'd like to prevent a mess similar to last summer ("release tomorrow no matter what!") and get GPU-features/cleanup ready on time. Thanks and best regards, Karli

Re: [petsc-dev] Request for comments on paper on PETSc TS

2018-02-21 Thread Karl Rupp
Hi, thanks, that's a nice manuscript! I like the general setup and flow of discussion. Here are a couple of further comments; feel free to consider or ignore as you see fit: Page 1: The references to MATLAB and NAG are incomplete, as they only show the year. Instead of "MATLAB [2014]" it

Re: [petsc-dev] CUDA white paper of interest

2018-02-14 Thread Karl Rupp
Hi Jonathan, thanks for your message and the pointer. The incomplete factorizations have been around for a while, and with recent hardware they tend to be less competitive (note that they use a Tesla 2050 in their benchmarks, which is ~7 years old). The fine-grained parallel version here:

Re: [petsc-dev] plans for preconditioners for SeqSELL

2018-02-12 Thread Karl Rupp
Hi Stefano, Is there any plan to write code for native ILU/ICC etc for SeqSELL, at least to have BJACOBI in parallel? (imho) ILU/ICC is a pain to do with SeqSELL. Point-Jacobi should be possible, yes. SELL is really just tailored to MatMults and a pain for anything that is not very similar

Re: [petsc-dev] why does veccuda.py exist?

2018-02-02 Thread Karl Rupp
Why can't the VECCUDA type coexist with the VECCUSP or VECVIENNACL types? If it can't coexist, can the code be reworked to allow it to coexist? Currently it can't coexist because some variables are conditionally compiled and may be multiply defined (e.g. spptr). Hmm, I don't think so.

Re: [petsc-dev] why does veccuda.py exist?

2018-02-02 Thread Karl Rupp
Hey, I'm am totally confused by 1) the existence of veccuda.py if I remember correctly, its purpose is to make sure that one of the GPU backends is enabled if a user configures --with-cuda. 2) the fact that veccuda.py depends on some packages but is not a package and is not in

Re: [petsc-dev] [SPAM?] Re: [SPAM *****] Re: Issue with Lapack names

2017-12-18 Thread Karl Rupp
> > This is related to a message I sent 2 years ago to petsc-maint "Inconsistent naming of one Lapack subroutine", where I advocated renaming LAPACKungqr_ --> LAPACKorgqr_. But that thread did not end up in any modification... > > > > I can't find the thread. I also

Re: [petsc-dev] PETSc Quarterly Telecon

2017-12-18 Thread Karl Rupp
be on a plane at that time so I won't be able to join. I would be grateful if someone could write some brief meeting minutes for those of us who are interested but cannot make it. Thanks, Richard On Mon, Dec 18, 2017 at 2:51 AM, Karl Rupp <r...@iue.tuwien.ac.at <mailto:r...@iue.tuwien

Re: [petsc-dev] PETSc Quarterly Telecon

2017-12-18 Thread Karl Rupp
Additional topic to discuss: * Minisymposium at SIAM Annual Meeting On 12/18/2017 11:51 AM, Karl Rupp wrote: Hi all, the PETSc quarterly telecon will take place today, Monday, at 2pm Chicago time. If anyone of you has a good software to hold the telecon (maybe BlueJeans or Zoom

Re: [petsc-dev] PETSc Quarterly Telecon

2017-12-18 Thread Karl Rupp
/13/2017 07:12 AM, Karl Rupp wrote: Dear PETSc folks, the next quarterly teleconference is due. It is intended to discuss "the bigger picture" rather than technical details. Thus, feel free to join even if you have not yet contributed thousands of lines of code (yet) ;-) Please

[petsc-dev] PETSc Quarterly Telecon

2017-12-12 Thread Karl Rupp
Dear PETSc folks, the next quarterly teleconference is due. It is intended to discuss "the bigger picture" rather than technical details. Thus, feel free to join even if you have not yet contributed thousands of lines of code (yet) ;-) Please state your availability by Sunday, December 17,

Re: [petsc-dev] Random123

2017-09-05 Thread Karl Rupp
Hi Toby, FYI: the Random123 license is just what is commonly referred to as the 2-clause BSD license: https://opensource.org/licenses/BSD-2-Clause I don't see any problem with making a repo for it and using it for a PetscRandom implementation (IANAL). Best regards, Karli On 09/05/2017

Re: [petsc-dev] Routines that change matrix entries without requiring another MatAssemblyBegin/End?

2017-07-30 Thread Karl Rupp
Well, I thought of AIJMKL keeping track of the state for which mkl_sparse_optimize() was called. If the matrix state changes, the next call to MatMult() is MatMult the only operation that needs this check, or do many Mat methods need this check? Looks like it is needed for MatMult(),

Re: [petsc-dev] Routines that change matrix entries without requiring another MatAssemblyBegin/End?

2017-07-29 Thread Karl Rupp
In theory PETSc Mat have two "state" values, 1) nonzerostate - this is increased anytime the nonzero structure changes 2) state - this is increased anytime any numerical values are changed These are used by PCSetUp() to determine if the preconditioner needs to be updated and if

Re: [petsc-dev] Routines that change matrix entries without requiring another MatAssemblyBegin/End?

2017-07-28 Thread Karl Rupp
Hey, In theory PETSc Mat have two "state" values, 1) nonzerostate - this is increased anytime the nonzero structure changes 2) state - this is increased anytime any numerical values are changed These are used by PCSetUp() to determine if the preconditioner needs to be updated

Re: [petsc-dev] GPU regression tests

2017-07-28 Thread Karl Rupp
Hi Alejandro, I have tested the branch and it seems to fix the problem. Our software runs fine with it. great, thanks for the quick feedback. Best regards, Karli

Re: [petsc-dev] GPU regression tests

2017-07-28 Thread Karl Rupp
(because it changes the pointers to MatMult_SeqAIJ_Inode etc). $ ./ex_duplicate -mat_type aijcusparse -vec_type cuda Thanks. Jose El 26 jul 2017, a las 21:12, Karl Rupp <r...@iue.tuwien.ac.at> escribió: Hi Jose, With pull request #719 we have finished a set of fixes to VECCUDA

Re: [petsc-dev] GPU regression tests

2017-07-27 Thread Karl Rupp
-double_es.log Satish On Thu, 27 Jul 2017, Karl Rupp wrote: Hi Jose, We have detected another problem. Could you take care of it? MatDuplicate() does not work for MATSEQAIJCUSPARSE (probably also for MATMPIAIJCUSPARSE). The attached example creates a matrix and duplicates it. There are two cases: 1

  1   2   3   4   5   6   >