Variable chunk size on the go?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Variable chunk size on the go?

peacherwu
Hi, I am new to hdf5. For my patching algorithm, I need to find the max matching subsequence between a portion of the old file and the new file. Then I need to encode the difference vector to a file with Gzip compressor. However, since the length of the difference vector is not fixed, is it suitable to use HDF5 API? I had a quick look at h5-cmprss.c and it seems to require fixed chunking.

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|

Re: Variable chunk size on the go?

Quincey Koziol-3
HDF5 has fixed sized chunks.  (Although we’ve thought about changing that)

        Quincey

> On May 19, 2017, at 9:48 AM, 武鹏程 <[hidden email]> wrote:
>
> Hi, I am new to hdf5. For my patching algorithm, I need to find the max matching subsequence between a portion of the old file and the new file. Then I need to encode the difference vector to a file with Gzip compressor. However, since the length of the difference vector is not fixed, is it suitable to use HDF5 API? I had a quick look at h5-cmprss.c and it seems to require fixed chunking.
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [hidden email]
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5