Unique object identifier

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Unique object identifier

Wintersberger, Eugen
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi folks
  I need to get a unique ID for each object in an HDF5 file (as the
path to an object is not unique). As far as I know there is nothing
like this in HDF5 out of the box. However, I have seen that one can
retrieve the address of an object within the file. Can this address be
used as a unique identifier?

regards
  Eugen
- --
- ---------------------------------------
| DI. Dr. Eugen Wintersberger         |
|                                     |
| FS-EC                               |
| HASYLAB at DESY                     |
| Notkestr. 85                        |
| D-22607 Hamburg                     |
| Germany                             |
|                                     |
| E-Mail: [hidden email] |
| Telefon: +49-40-8998-1917           |
- ---------------------------------------
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iF4EAREIAAYFAlAuRvMACgkQlIP0H82W0vCJTwD/Ze5+eaPKrinrNPGwRLGPYHJR
4k9fjAeymXEm6Q3D6skA/jzIKsqxqU5igtqlcK0Ouei8Ee02DtKBqgI/CSpEUizW
=ktNR
-----END PGP SIGNATURE-----

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Reply | Threaded
Open this post in threaded view
|

Re: Unique object identifier

Gerd Heber
Eugen, how are you? Yes, you can concoct something that is unique in the
file scope using addresses, but I'd be hesitant to go down that path.
I'd invest in a few UUIDs (call it whatever you want) attributes and not
look back. They survive repacks and other machinations.
True, for existing read-only files, you'd have to maintain a mapping on the
side.
Addresses are not your friends, if you know what I mean...

Best, G.

-----Original Message-----
From: [hidden email] [mailto:[hidden email]]
On Behalf Of Wintersberger, Eugen
Sent: Friday, August 17, 2012 8:28 AM
To: [hidden email]
Subject: [Hdf-forum] Unique object identifier

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi folks
  I need to get a unique ID for each object in an HDF5 file (as the path to
an object is not unique). As far as I know there is nothing like this in
HDF5 out of the box. However, I have seen that one can retrieve the address
of an object within the file. Can this address be used as a unique
identifier?

regards
  Eugen
- --
- ---------------------------------------
| DI. Dr. Eugen Wintersberger         |
|                                     |
| FS-EC                               |
| HASYLAB at DESY                     |
| Notkestr. 85                        |
| D-22607 Hamburg                     |
| Germany                             |
|                                     |
| E-Mail: [hidden email] |
| Telefon: +49-40-8998-1917           |
- ---------------------------------------
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iF4EAREIAAYFAlAuRvMACgkQlIP0H82W0vCJTwD/Ze5+eaPKrinrNPGwRLGPYHJR
4k9fjAeymXEm6Q3D6skA/jzIKsqxqU5igtqlcK0Ouei8Ee02DtKBqgI/CSpEUizW
=ktNR
-----END PGP SIGNATURE-----

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org



_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Reply | Threaded
Open this post in threaded view
|

Re: Unique object identifier

Wintersberger, Eugen
Thanks for the fast reply. I do not UUIDs to survive anything. In fact
they should only be unique as long as a file is open. I am working on a
thin, thread-safe C++ wrapper around HDF5. Locking the individual
objects and make their interfaces thread-safe is one thing. But there is
another problem with HDF5: consider two concurrent threads opening the
same object in the file. Though the types returned would be thread-safe,
the fact that both threads act on the same object in the file would lead
to inconsistencies. To circumvent this problem I plan to establish a
central registry where all objects are retrieved from. Each object is
opened only once and appended to the registry using a unique identifier
as a key (that's why I need a unique identifier for each object).
Clearly these UUIDs must be only unique during the lifetime of a file
object. The code itself only works with references to the objects stored
in the registry. Thus, at least I think so, locking the objects should
make sense again and two threads can safely work on the same object.

regards
  Eugen



On Fri, 2012-08-17 at 08:41 -0500, Gerd Heber wrote:

> Eugen, how are you? Yes, you can concoct something that is unique in the
> file scope using addresses, but I'd be hesitant to go down that path.
> I'd invest in a few UUIDs (call it whatever you want) attributes and not
> look back. They survive repacks and other machinations.
> True, for existing read-only files, you'd have to maintain a mapping on the
> side.
> Addresses are not your friends, if you know what I mean...
>
> Best, G.
>
> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]]
> On Behalf Of Wintersberger, Eugen
> Sent: Friday, August 17, 2012 8:28 AM
> To: [hidden email]
> Subject: [Hdf-forum] Unique object identifier
>
> Hi folks
>   I need to get a unique ID for each object in an HDF5 file (as the path to
> an object is not unique). As far as I know there is nothing like this in
> HDF5 out of the box. However, I have seen that one can retrieve the address
> of an object within the file. Can this address be used as a unique
> identifier?
>
> regards
>   Eugen
> --
> ---------------------------------------
> | DI. Dr. Eugen Wintersberger         |
> |                                     |
> | FS-EC                               |
> | HASYLAB at DESY                     |
> | Notkestr. 85                        |
> | D-22607 Hamburg                     |
> | Germany                             |
> |                                     |
> | E-Mail: [hidden email] |
> | Telefon: +49-40-8998-1917           |
> ---------------------------------------
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [hidden email]
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [hidden email]
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

signature.asc (237 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Unique object identifier

Ruth Aydt
Administrator
Re: thread-safe access to HDF5 objects, if you haven't already done so, I recommend you take a look at these posts in the FAQ.    There are some subtleties because of shared data structures in the HDF5 library that aren't immediately obvious.  

http://www.hdfgroup.org/hdf5-quest.html#gconc

http://www.hdfgroup.org/hdf5-quest.html#tsafe

It wasn't clear to me from your post if you were planning to use a thread-safe build of HDF5 or not.

--Ruth
Reply | Threaded
Open this post in threaded view
|

Re: Unique object identifier

Quincey Koziol
In reply to this post by Wintersberger, Eugen
Hi Eugen,
        HDF5 will maintain the consistency of the objects opened via two different threads internal to the library.  Modifications that one thread makes to the object may make "derived" information in the other thread inconsistent, but the library's perspective will be fine.

        Quincey

On Aug 18, 2012, at 7:27 AM, Wintersberger, Eugen wrote:

> Thanks for the fast reply. I do not UUIDs to survive anything. In fact
> they should only be unique as long as a file is open. I am working on a
> thin, thread-safe C++ wrapper around HDF5. Locking the individual
> objects and make their interfaces thread-safe is one thing. But there is
> another problem with HDF5: consider two concurrent threads opening the
> same object in the file. Though the types returned would be thread-safe,
> the fact that both threads act on the same object in the file would lead
> to inconsistencies. To circumvent this problem I plan to establish a
> central registry where all objects are retrieved from. Each object is
> opened only once and appended to the registry using a unique identifier
> as a key (that's why I need a unique identifier for each object).
> Clearly these UUIDs must be only unique during the lifetime of a file
> object. The code itself only works with references to the objects stored
> in the registry. Thus, at least I think so, locking the objects should
> make sense again and two threads can safely work on the same object.
>
> regards
>  Eugen
>
>
>
> On Fri, 2012-08-17 at 08:41 -0500, Gerd Heber wrote:
>> Eugen, how are you? Yes, you can concoct something that is unique in the
>> file scope using addresses, but I'd be hesitant to go down that path.
>> I'd invest in a few UUIDs (call it whatever you want) attributes and not
>> look back. They survive repacks and other machinations.
>> True, for existing read-only files, you'd have to maintain a mapping on the
>> side.
>> Addresses are not your friends, if you know what I mean...
>>
>> Best, G.
>>
>> -----Original Message-----
>> From: [hidden email] [mailto:[hidden email]]
>> On Behalf Of Wintersberger, Eugen
>> Sent: Friday, August 17, 2012 8:28 AM
>> To: [hidden email]
>> Subject: [Hdf-forum] Unique object identifier
>>
>> Hi folks
>>  I need to get a unique ID for each object in an HDF5 file (as the path to
>> an object is not unique). As far as I know there is nothing like this in
>> HDF5 out of the box. However, I have seen that one can retrieve the address
>> of an object within the file. Can this address be used as a unique
>> identifier?
>>
>> regards
>>  Eugen
>> --
>> ---------------------------------------
>> | DI. Dr. Eugen Wintersberger         |
>> |                                     |
>> | FS-EC                               |
>> | HASYLAB at DESY                     |
>> | Notkestr. 85                        |
>> | D-22607 Hamburg                     |
>> | Germany                             |
>> |                                     |
>> | E-Mail: [hidden email] |
>> | Telefon: +49-40-8998-1917           |
>> ---------------------------------------
>> _______________________________________________
>> Hdf-forum is for HDF software users discussion.
>> [hidden email]
>> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>>
>>
>>
>> _______________________________________________
>> Hdf-forum is for HDF software users discussion.
>> [hidden email]
>> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [hidden email]
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org