implementing the attribute callbakcs in a VOL plugin

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

implementing the attribute callbakcs in a VOL plugin

dimstamat
Thanks Mohamad, I managed to do that two weeks ago and it worked perfectly! I serialized the data structure of the datatype and dataspace and now there are no issues. Thanks for the help!


Dimos

---
Dimokritos Stamatakis
PhD student,
Brandeis University

On Jun 14, 2016, at 10:27 AM, [hidden email] wrote:

Send Hdf-forum mailing list submissions to
[hidden email]

To subscribe or unsubscribe via the World Wide Web, visit
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

or, via email, send a message with subject or body 'help' to
[hidden email]

You can reach the person managing the list at
[hidden email]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Hdf-forum digest..."


Today's Topics:

  1. Re: group/dataset sequence (?????? ?????????)
  2. Re: Parallel HDF5 file locking error (Mohamad Chaarawi)
  3. Re: implementing the attribute callbakcs in a VOL plugin
     (Mohamad Chaarawi)


----------------------------------------------------------------------

Message: 1
Date: Tue, 14 Jun 2016 16:02:26 +0300
From: ?????? ????????? <[hidden email]>
To: [hidden email]
Subject: Re: [Hdf-forum] group/dataset sequence
Message-ID: <[hidden email]>
Content-Type: text/plain; charset=UTF-8; format=flowed

14.06.2016 4:02, ??? ?????:
I am a newer to HDF5. I have a question, and didn't find the solution on
internet. Could you help me? Thanks.

I create three group named "fileHead" "auxiliaryInfo" "observationData"
in sequence. But the HDFView show is  "auxiliaryInfo" "fileHead"
"observationData" by alphabet. I want to show by the creation sequence.
So what should I do when I create these groups?
The dataset is also has this phenomenon.

Hello ???!

This documentation page should be of help for you:
https://www.hdfgroup.org/HDF5/doc/RM/RM_H5P.html#Property-SetLinkCreationOrder

Best wishes,
Andrey Paramonov


--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.




------------------------------

Message: 2
Date: Tue, 14 Jun 2016 14:25:09 +0000
From: Mohamad Chaarawi <[hidden email]>
To: HDF Users Discussion List <[hidden email]>
Subject: Re: [Hdf-forum] Parallel HDF5 file locking error
Message-ID: <[hidden email]>
Content-Type: text/plain; charset="utf-8"

Hi Andrew,

Since it?s going to be hard to replicate this, could you provide more information by attaching a debugger and show the trace of where the hang is?

Thanks,
Mohamad

From: Hdf-forum <[hidden email]> on behalf of Andrew Ho <[hidden email]>
Reply-To: hdf-forum <[hidden email]>
Date: Friday, June 10, 2016 at 10:06 PM
To: hdf-forum <[hidden email]>
Subject: [Hdf-forum] Parallel HDF5 file locking error

I'm having some issues with file locking on a parallel filesystem which I believe is related the problem here: https://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2011-February/004254.html

I tried the suggestion of disabling romio_ds_read and romio_ds_write, but this doesn't fix the problem. I've also tried setting H5Pset_seive_buf_size to 0 to disable data striding in HDF5 itself, but that doesn't work, either.

Here's a short snippet of the relevant code:

hid_t fapl_id = H5Pcreate(H5P_FILE_ACCESS);
MPI_Info info;
MPI_Info_create(&info);
MPI_Info_set(info, "romio_ds_read", "disable");
MPI_Info_set(info, "romio_ds_write", "disable");
H5Pset_sieve_buf_size(fapl_id, 0);
H5Pset_fapl_mpio(fapl_id, comm, info);

hid_t f_id = H5Fcreate("test_file.h5", H5F_ACC_TRUNC, H5P_DEFAULT, fapl_id);

It's on this last line that the program hangs, and eventually MPI_ABORT gets called. Note that I get the same problem regardless of if I try to create a new file, or if I try to open an existing file:

hid_t f_id = H5Fopen("test_file.h5", H5F_ACC_RDWR, fapl_id);

Other relevant information:

1. Filesystem is NFS v3 with noac off (can't change)
2. Tested with HDF 5.8.13 and 5.8.14
3. Tested with OpenMPI 1.6.5 and 1.10.2

What am I missing? I saw that there's an option to disable filesystem atomicity, but this requires that the file already be opened, and I can't even get that far.

I also know that my code does work on different computers with different filesystems and/or mount options.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/attachments/20160614/a81fc4c2/attachment-0001.html>

------------------------------

Message: 3
Date: Tue, 14 Jun 2016 14:27:12 +0000
From: Mohamad Chaarawi <[hidden email]>
To: HDF Users Discussion List <[hidden email]>
Subject: Re: [Hdf-forum] implementing the attribute callbakcs in a VOL
plugin
Message-ID: <[hidden email]>
Content-Type: text/plain; charset="utf-8"

Usually in custom VOL plugin, the easiest way to store the datatype and dataspace for an attribute or dataset is to serialize them and store them as a blob.
Look into the function H5Tencode/decode() and H5Sencode/decode().

Thanks,
Mohamad

From: Hdf-forum <[hidden email]> on behalf of Dimos Stamatakis <[hidden email]>
Reply-To: hdf-forum <[hidden email]>
Date: Friday, May 27, 2016 at 1:20 AM
To: hdf-forum <[hidden email]>
Subject: [Hdf-forum] implementing the attribute callbakcs in a VOL plugin

Hi all,

Regarding my previous question about an extra call of H5Fclose, there was indeed an extra open handle. Now everything works perfectly!
Now I have a question about the attribute callbacks of a VOL plugin and the datatype. I am guessing on attribute create we have to store the specified datatype and dataspace using the custom VOL plugin. The question is how will we restore them, since for example the datatype has to be in the HDF5 format (H5T_t type), it is not something custom like the VOL objects. I saw that calling H5I_register() on a datatype generates a location ID, but it is not working properly.
What is the correct procedure of implementing attribute callbacks for a VOL plugin? Also I saw that the datatype callbacks are not invoked in the native HDF5 when creating and accessing string attributes.
For example, when creating a string attribute, closing it, then opening it again and getting its datatype it should return a location ID of its datatype. This is quite confusing, since how will the VOL plugin create these HDF5 specific objects?

Thanks,

Dimos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/attachments/20160614/493b516a/attachment.html>

------------------------------

Subject: Digest Footer

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org


------------------------------

End of Hdf-forum Digest, Vol 84, Issue 10
*****************************************


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Loading...