Transposing a 2d matrix via hyperslab

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Transposing a 2d matrix via hyperslab

Wei-keng Liao

In NetCDF/PnetCDF, there is an API family named varm that can achieve such effect.

APIs nc_put_varm_int/ncmpi_put_varm_int allows users to writes a mapped
array section of values. The argument imap can be used to describe the "transpose"
layout of the write buffer. See their API references in
http://cucis.ece.northwestern.edu/projects/PnetCDF/doc/pnetcdf-c/ncmpi_005fput_005fvarm_005f_003ctype_003e.html
http://www.unidata.ucar.edu/software/netcdf/docs/group__variables.html#ga3ff88687cfc7b58293a1cc883780b453

In PnetCDF internal implementation, the write buffer is first transposed to a temporary
buffer which is then written to file. The in-memory 2D transpose is done by first
creating an MPI derived datatype based on the imap argument, followed by a call to
MPI_Pack to "pack" the write buffer to the temporary buffer.

So, the only subroutine an HDF5 user program needs is that in-memory transpose.
I can extract such subroutine from PnetCDF, but it will be using MPI. Let me know.

If you are interested in the PnetCDF case, an example C program is available in
http://trac.mcs.anl.gov/projects/parallel-netcdf/browser/trunk/examples/C/transpose.c
and a short description in
http://trac.mcs.anl.gov/projects/parallel-netcdf/browser/trunk/examples/README


Wei-keng

> A coworker and I are mulling over if its possible to take a simple 3x3
> matrix (stored row major) in a dataspace and write it out transposed to
> disk using a hyperslab.   It seems that the grammer provided by start,
> stride, blocks, and count won't allow for this.  We hope we are wrong!  It
> is possible to do this?
>
> Thanks,
> Isaac


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|

Re: Transposing a 2d matrix via hyperslab

Isaac Gerg
Thanks for the reply, but its really easy for us to do the transpose in memory.  -Isaac

On Sat, Apr 29, 2017 at 6:20 PM, Wei-keng Liao <[hidden email]> wrote:

In NetCDF/PnetCDF, there is an API family named varm that can achieve such effect.

APIs nc_put_varm_int/ncmpi_put_varm_int allows users to writes a mapped
array section of values. The argument imap can be used to describe the "transpose"
layout of the write buffer. See their API references in
http://cucis.ece.northwestern.edu/projects/PnetCDF/doc/pnetcdf-c/ncmpi_005fput_005fvarm_005f_003ctype_003e.html
http://www.unidata.ucar.edu/software/netcdf/docs/group__variables.html#ga3ff88687cfc7b58293a1cc883780b453

In PnetCDF internal implementation, the write buffer is first transposed to a temporary
buffer which is then written to file. The in-memory 2D transpose is done by first
creating an MPI derived datatype based on the imap argument, followed by a call to
MPI_Pack to "pack" the write buffer to the temporary buffer.

So, the only subroutine an HDF5 user program needs is that in-memory transpose.
I can extract such subroutine from PnetCDF, but it will be using MPI. Let me know.

If you are interested in the PnetCDF case, an example C program is available in
http://trac.mcs.anl.gov/projects/parallel-netcdf/browser/trunk/examples/C/transpose.c
and a short description in
http://trac.mcs.anl.gov/projects/parallel-netcdf/browser/trunk/examples/README


Wei-keng

> A coworker and I are mulling over if its possible to take a simple 3x3
> matrix (stored row major) in a dataspace and write it out transposed to
> disk using a hyperslab.   It seems that the grammer provided by start,
> stride, blocks, and count won't allow for this.  We hope we are wrong!  It
> is possible to do this?
>
> Thanks,
> Isaac


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5