Quantcast

More informative error from H5Fopen?

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

More informative error from H5Fopen?

Elvis Stansvik
Hi all,

Is it possible to get some more informative error from H5Fopen than
"it failed"? E.g. if it was a permission problem, nonexistent file,
corrupted file, ...? Or must I try to deduce the reason for the
failure myself? (always risky to do such checks pre- or post-mortem so
to speak..).

Thanks in advance,
Elvis

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: More informative error from H5Fopen?

Rob Latham


On 11/08/2016 01:18 PM, Elvis Stansvik wrote:
> Hi all,
>
> Is it possible to get some more informative error from H5Fopen than
> "it failed"? E.g. if it was a permission problem, nonexistent file,
> corrupted file, ...? Or must I try to deduce the reason for the
> failure myself? (always risky to do such checks pre- or post-mortem so
> to speak..).

I think you are going to have to share your output with the list.
Whenever *I* run hdf5 and get an error I get screenfuls of information.

For example, here's a simple hdf5 program trying to open a file for
which I have removed all read and write permissions:

HDF5-DIAG: Error detected in HDF5 (1.8.16) thread 0:
   #000: ../../../hdf5-1.8.16/src/H5F.c line 604 in H5Fopen(): unable to
open file
     major: File accessibilty
     minor: Unable to open file
   #001: ../../../hdf5-1.8.16/src/H5Fint.c line 992 in H5F_open():
unable to open file: time = Mon Nov 28 14:41:40 2016
, name = 'try.h5', tent_flags = 1
     major: File accessibilty
     minor: Unable to open file
   #002: ../../../hdf5-1.8.16/src/H5FD.c line 993 in H5FD_open(): open
failed
     major: Virtual File Layer
     minor: Unable to initialize object
   #003: ../../../hdf5-1.8.16/src/H5FDsec2.c line 339 in
H5FD_sec2_open(): unable to open file: name = 'try.h5', errno = 13,
error message = 'Permission denied', flags = 1, o_flags = 2
     major: File accessibilty
     minor: Unable to open file

>
> Thanks in advance,
> Elvis
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [hidden email]
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> Twitter: https://twitter.com/hdf5
>

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: More informative error from H5Fopen?

Miller, Mark C.
"Hdf-forum on behalf of Rob Latham" wrote:


I think you are going to have to share your output with the list.
Whenever *I* run hdf5 and get an error I get screenfuls of information.

For example, here's a simple hdf5 program trying to open a file for
which I have removed all read and write permissions:

HDF5-DIAG: Error detected in HDF5 (1.8.16) thread 0:
   #000: ../../../hdf5-1.8.16/src/H5F.c line 604 in H5Fopen(): unable to
open file
     major: File accessibilty
     minor: Unable to open file
   #001: ../../../hdf5-1.8.16/src/H5Fint.c line 992 in H5F_open():
unable to open file: time = Mon Nov 28 14:41:40 2016
, name = 'try.h5', tent_flags = 1
     major: File accessibilty
     minor: Unable to open file
   #002: ../../../hdf5-1.8.16/src/H5FD.c line 993 in H5FD_open(): open
failed
     major: Virtual File Layer
     minor: Unable to initialize object
   #003: ../../../hdf5-1.8.16/src/H5FDsec2.c line 339 in
H5FD_sec2_open(): unable to open file: name = 'try.h5', errno = 13,
error message = 'Permission denied', flags = 1, o_flags = 2
     major: File accessibilty
     minor: Unable to open file


I am not sure how other users feel about this and it may not even be the original poster's ( Elvis Stansvik) complaint but a big challenge I have with interpreting HDF5 errors *is* the "...screenfuls of information".

It is challenging to paw through it to discover the root cause.

For example, in the above error stack, we get information about why sec2 VFD failed..."errno=13 ("Permission enied") but what are 'flags' and 'o_flags' there? Are these params to sec2 open call? That has only an 'oflag' argument and not an additional 'flags' argument. And, their numeric values, '1' and '2' are too informatvie either. What about similar info at the HDF5 public interface (e.g. H5Fopen)? What was passed for 'flags' there. Oh, maybe 'flags' in the #003 trace *is* the 'flags' argument caller passed to H5Fopen and 'o_flags' is what HDF5 lib passed to sec2 open call? I could see having 'flags' emitted in the #000 trace if that is indeed the argument passed as 'flags' to H5Fopen. and 'oflags' not 'o_flags' emitted in the #003 trace. Finally, it would be *much* more helpful to see the symbolic constants for the flags and not their numerics.

So, I personally find grocking HDF5 error stack output very awkward. 

It may be that it is awkward enough for others that no one really is able to find the information they needed in it (e.g. the signal to noise ratio is just way too low).

My two cents.

Mark

-- 
Mark C. Miller, LLNL

"In the end, we will remember not the words of
our enemies but the silence of our friends" - MLK

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: More informative error from H5Fopen?

Landon Clipp

Mark,

I get what you mean. I have rarely found the error output of HDF5 to be of much use, although sometimes it can be helpful. For instance just off the top of my head, I've been getting errors about HDF attributes. I've called H5Aclose and I get errors about the argument not being an attribute. Turns out the variable I passed was never initialized. So in that case it's helpful.

One thing to realize is that the error is most likely NOT reading in the library itself. I recommend that you use a debugger like GDB to find exactly what you are passing into the function and to see if every value you're passing in makes sense. That's really the only way to go about it.

It all comes down to this: H5Fopen is failing and there are a finite number of issues we can address.

1. Does the file exist, and if so are you providing the correct path to it (relative or absolute)? Do you have read/write permission for the file?
2. Have you misspelled the file?
3. Are you providing either DF_ACC_RDWR or DF_ACC_RDONLY to the second argument? Only those two arguments are valid.
4. What file access property list are you providing? You should be able to rule out providing a buggy access property by just passing in the default parameter.

I guarantee that your error lies in one of these issues. I recommend you make the second and third arguments the read only and default respectively. That will isolate the problem to your first argument and whether or not you have appropriate file permissions.

Regards,
Landon


On Nov 28, 2016 3:33 PM, "Miller, Mark C." <[hidden email]> wrote:
"Hdf-forum on behalf of Rob Latham" wrote:


I think you are going to have to share your output with the list.
Whenever *I* run hdf5 and get an error I get screenfuls of information.

For example, here's a simple hdf5 program trying to open a file for
which I have removed all read and write permissions:

HDF5-DIAG: Error detected in HDF5 (1.8.16) thread 0:
   #000: ../../../hdf5-1.8.16/src/H5F.c line 604 in H5Fopen(): unable to
open file
     major: File accessibilty
     minor: Unable to open file
   #001: ../../../hdf5-1.8.16/src/H5Fint.c line 992 in H5F_open():
unable to open file: time = Mon Nov 28 14:41:40 2016
, name = 'try.h5', tent_flags = 1
     major: File accessibilty
     minor: Unable to open file
   #002: ../../../hdf5-1.8.16/src/H5FD.c line 993 in H5FD_open(): open
failed
     major: Virtual File Layer
     minor: Unable to initialize object
   #003: ../../../hdf5-1.8.16/src/H5FDsec2.c line 339 in
H5FD_sec2_open(): unable to open file: name = 'try.h5', errno = 13,
error message = 'Permission denied', flags = 1, o_flags = 2
     major: File accessibilty
     minor: Unable to open file


I am not sure how other users feel about this and it may not even be the original poster's ( Elvis Stansvik) complaint but a big challenge I have with interpreting HDF5 errors *is* the "...screenfuls of information".

It is challenging to paw through it to discover the root cause.

For example, in the above error stack, we get information about why sec2 VFD failed..."errno=13 ("Permission enied") but what are 'flags' and 'o_flags' there? Are these params to sec2 open call? That has only an 'oflag' argument and not an additional 'flags' argument. And, their numeric values, '1' and '2' are too informatvie either. What about similar info at the HDF5 public interface (e.g. H5Fopen)? What was passed for 'flags' there. Oh, maybe 'flags' in the #003 trace *is* the 'flags' argument caller passed to H5Fopen and 'o_flags' is what HDF5 lib passed to sec2 open call? I could see having 'flags' emitted in the #000 trace if that is indeed the argument passed as 'flags' to H5Fopen. and 'oflags' not 'o_flags' emitted in the #003 trace. Finally, it would be *much* more helpful to see the symbolic constants for the flags and not their numerics.

So, I personally find grocking HDF5 error stack output very awkward. 

It may be that it is awkward enough for others that no one really is able to find the information they needed in it (e.g. the signal to noise ratio is just way too low).

My two cents.

Mark

-- 
Mark C. Miller, LLNL

"In the end, we will remember not the words of
our enemies but the silence of our friends" - MLK

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: More informative error from H5Fopen?

Landon Clipp

Also one more possible issue I'd add onto my list there: it is possible that the file is corrupt. So try ruling that out by making an empty HDF5 file in the same directory as the one you're trying to access and run the same functions, only opening this new file instead.

Regards,
Landon


On Nov 28, 2016 4:26 PM, "Landon Clipp" <[hidden email]> wrote:

Mark,

I get what you mean. I have rarely found the error output of HDF5 to be of much use, although sometimes it can be helpful. For instance just off the top of my head, I've been getting errors about HDF attributes. I've called H5Aclose and I get errors about the argument not being an attribute. Turns out the variable I passed was never initialized. So in that case it's helpful.

One thing to realize is that the error is most likely NOT reading in the library itself. I recommend that you use a debugger like GDB to find exactly what you are passing into the function and to see if every value you're passing in makes sense. That's really the only way to go about it.

It all comes down to this: H5Fopen is failing and there are a finite number of issues we can address.

1. Does the file exist, and if so are you providing the correct path to it (relative or absolute)? Do you have read/write permission for the file?
2. Have you misspelled the file?
3. Are you providing either DF_ACC_RDWR or DF_ACC_RDONLY to the second argument? Only those two arguments are valid.
4. What file access property list are you providing? You should be able to rule out providing a buggy access property by just passing in the default parameter.

I guarantee that your error lies in one of these issues. I recommend you make the second and third arguments the read only and default respectively. That will isolate the problem to your first argument and whether or not you have appropriate file permissions.

Regards,
Landon


On Nov 28, 2016 3:33 PM, "Miller, Mark C." <[hidden email]> wrote:
"Hdf-forum on behalf of Rob Latham" wrote:


I think you are going to have to share your output with the list.
Whenever *I* run hdf5 and get an error I get screenfuls of information.

For example, here's a simple hdf5 program trying to open a file for
which I have removed all read and write permissions:

HDF5-DIAG: Error detected in HDF5 (1.8.16) thread 0:
   #000: ../../../hdf5-1.8.16/src/H5F.c line 604 in H5Fopen(): unable to
open file
     major: File accessibilty
     minor: Unable to open file
   #001: ../../../hdf5-1.8.16/src/H5Fint.c line 992 in H5F_open():
unable to open file: time = Mon Nov 28 14:41:40 2016
, name = 'try.h5', tent_flags = 1
     major: File accessibilty
     minor: Unable to open file
   #002: ../../../hdf5-1.8.16/src/H5FD.c line 993 in H5FD_open(): open
failed
     major: Virtual File Layer
     minor: Unable to initialize object
   #003: ../../../hdf5-1.8.16/src/H5FDsec2.c line 339 in
H5FD_sec2_open(): unable to open file: name = 'try.h5', errno = 13,
error message = 'Permission denied', flags = 1, o_flags = 2
     major: File accessibilty
     minor: Unable to open file


I am not sure how other users feel about this and it may not even be the original poster's ( Elvis Stansvik) complaint but a big challenge I have with interpreting HDF5 errors *is* the "...screenfuls of information".

It is challenging to paw through it to discover the root cause.

For example, in the above error stack, we get information about why sec2 VFD failed..."errno=13 ("Permission enied") but what are 'flags' and 'o_flags' there? Are these params to sec2 open call? That has only an 'oflag' argument and not an additional 'flags' argument. And, their numeric values, '1' and '2' are too informatvie either. What about similar info at the HDF5 public interface (e.g. H5Fopen)? What was passed for 'flags' there. Oh, maybe 'flags' in the #003 trace *is* the 'flags' argument caller passed to H5Fopen and 'o_flags' is what HDF5 lib passed to sec2 open call? I could see having 'flags' emitted in the #000 trace if that is indeed the argument passed as 'flags' to H5Fopen. and 'oflags' not 'o_flags' emitted in the #003 trace. Finally, it would be *much* more helpful to see the symbolic constants for the flags and not their numerics.

So, I personally find grocking HDF5 error stack output very awkward. 

It may be that it is awkward enough for others that no one really is able to find the information they needed in it (e.g. the signal to noise ratio is just way too low).

My two cents.

Mark

-- 
Mark C. Miller, LLNL

"In the end, we will remember not the words of
our enemies but the silence of our friends" - MLK

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: More informative error from H5Fopen?

Rob Latham


On 11/28/2016 04:28 PM, Landon Clipp wrote:
> Also one more possible issue I'd add onto my list there: it is possible
> that the file is corrupt. So try ruling that out by making an empty HDF5
> file in the same directory as the one you're trying to access and run
> the same functions, only opening this new file instead.

I think Mark and Landon are saying the same things here, but in
different ways.

The 300-some HDF5 user-facing interfaces should return something
helpful.  The internal interfaces between, say, H5Fopen  and
lowest-level posix open(2) system call are not helpful for users.  The
probably aren't all that helpful for developers at this point in the
HDF5 life cycle, either.

==rob

>
> Regards,
> Landon
>
>
> On Nov 28, 2016 4:26 PM, "Landon Clipp" <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Mark,
>
>     I get what you mean. I have rarely found the error output of HDF5 to
>     be of much use, although sometimes it can be helpful. For instance
>     just off the top of my head, I've been getting errors about HDF
>     attributes. I've called H5Aclose and I get errors about the argument
>     not being an attribute. Turns out the variable I passed was never
>     initialized. So in that case it's helpful.
>
>     One thing to realize is that the error is most likely NOT reading in
>     the library itself. I recommend that you use a debugger like GDB to
>     find exactly what you are passing into the function and to see if
>     every value you're passing in makes sense. That's really the only
>     way to go about it.
>
>     It all comes down to this: H5Fopen is failing and there are a finite
>     number of issues we can address.
>
>     1. Does the file exist, and if so are you providing the correct path
>     to it (relative or absolute)? Do you have read/write permission for
>     the file?
>     2. Have you misspelled the file?
>     3. Are you providing either DF_ACC_RDWR or DF_ACC_RDONLY to the
>     second argument? Only those two arguments are valid.
>     4. What file access property list are you providing? You should be
>     able to rule out providing a buggy access property by just passing
>     in the default parameter.
>
>     I guarantee that your error lies in one of these issues. I recommend
>     you make the second and third arguments the read only and default
>     respectively. That will isolate the problem to your first argument
>     and whether or not you have appropriate file permissions.
>
>     Regards,
>     Landon
>
>
>     On Nov 28, 2016 3:33 PM, "Miller, Mark C." <[hidden email]
>     <mailto:[hidden email]>> wrote:
>
>         "Hdf-forum on behalf of Rob Latham" wrote:
>
>
>             I think you are going to have to share your output with the
>             list.
>             Whenever *I* run hdf5 and get an error I get screenfuls of
>             information.
>
>             For example, here's a simple hdf5 program trying to open a
>             file for
>             which I have removed all read and write permissions:
>
>             HDF5-DIAG: Error detected in HDF5 (1.8.16) thread 0:
>                #000: ../../../hdf5-1.8.16/src/H5F.c line 604 in
>             H5Fopen(): unable to
>             open file
>                  major: File accessibilty
>                  minor: Unable to open file
>                #001: ../../../hdf5-1.8.16/src/H5Fint.c line 992 in
>             H5F_open():
>             unable to open file: time = Mon Nov 28 14:41:40 2016
>             , name = 'try.h5', tent_flags = 1
>                  major: File accessibilty
>                  minor: Unable to open file
>                #002: ../../../hdf5-1.8.16/src/H5FD.c line 993 in
>             H5FD_open(): open
>             failed
>                  major: Virtual File Layer
>                  minor: Unable to initialize object
>                #003: ../../../hdf5-1.8.16/src/H5FDsec2.c line 339 in
>             H5FD_sec2_open(): unable to open file: name = 'try.h5',
>             errno = 13,
>             error message = 'Permission denied', flags = 1, o_flags = 2
>                  major: File accessibilty
>                  minor: Unable to open file
>
>
>
>         I am not sure how other users feel about this and it may not
>         even be the original poster's ( Elvis Stansvik) complaint but a
>         big challenge I have with interpreting HDF5 errors *is* the
>         "...screenfuls of information".
>
>         It is challenging to paw through it to discover the root cause.
>
>         For example, in the above error stack, we get information about
>         why sec2 VFD failed..."errno=13 ("Permission enied") but what
>         are 'flags' and 'o_flags' there? Are these params to sec2 open
>         call? That has only an 'oflag' argument and not an additional
>         'flags' argument. And, their numeric values, '1' and '2' are too
>         informatvie either. What about similar info at the HDF5 public
>         interface (e.g. H5Fopen)? What was passed for 'flags' there. Oh,
>         maybe 'flags' in the #003 trace *is* the 'flags' argument caller
>         passed to H5Fopen and 'o_flags' is what HDF5 lib passed to sec2
>         open call? I could see having 'flags' emitted in the #000 trace
>         if that is indeed the argument passed as 'flags' to H5Fopen. and
>         'oflags' not 'o_flags' emitted in the #003 trace. Finally, it
>         would be *much* more helpful to see the symbolic constants for
>         the flags and not their numerics.
>
>         So, I personally find grocking HDF5 error stack output very
>         awkward.
>
>         It may be that it is awkward enough for others that no one
>         really is able to find the information they needed in it (e.g.
>         the signal to noise ratio is just way too low).
>
>         My two cents.
>
>         Mark
>
>         --
>         Mark C. Miller, LLNL
>
>         "In the end, we will remember not the words of
>         our enemies but the silence of our friends" - MLK
>
>         _______________________________________________
>         Hdf-forum is for HDF software users discussion.
>         [hidden email] <mailto:[hidden email]>
>         http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>         <http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org>
>         Twitter: https://twitter.com/hdf5
>
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [hidden email]
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> Twitter: https://twitter.com/hdf5
>

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: More informative error from H5Fopen?

Elvis Stansvik
In reply to this post by Rob Latham
Hi Rob,

2016-11-28 21:43 GMT+01:00 Rob Latham <[hidden email]>:


On 11/08/2016 01:18 PM, Elvis Stansvik wrote:
Hi all,

Is it possible to get some more informative error from H5Fopen than
"it failed"? E.g. if it was a permission problem, nonexistent file,
corrupted file, ...? Or must I try to deduce the reason for the
failure myself? (always risky to do such checks pre- or post-mortem so
to speak..).

I think you are going to have to share your output with the list. Whenever *I* run hdf5 and get an error I get screenfuls of information.

For example, here's a simple hdf5 program trying to open a file for which I have removed all read and write permissions:

Yes, and I'm sorry. I should have replied to myself quite soon after I sent my message, but somehow forgot about it.

When I first wrote my message, I was

  1. working through the C++ API, and
  2. building my program in release mode.

Working through the C++ API meant I was using H5::Exception::getDetailMsg(), hoping to get some detailed message, but it was giving me something like "H5Fopen failed" (which is why I mentioned the H5Fopen C function in my post). Building my program in release mode also meant I did not get the detailed error stack printout that the C layer does by default in debug mode (like you showed below). That's why I was a little puzzled.

Since I wrote my message, I have

  1. switched to working with the C API (through a simple HDF5File C++ class of my own),
  2. learned all about the gory details of the H5E API [1] and error handling in chapter 9 of the User Guide.

So I'm all good now :) Though I must say the error handling API is quite complicated (I still haven't written my custom error walking/formatting functions, so only relying on the default printouts I get in debug mode).

One thing that I would wish for is

  1. an API similar to H5Eprint that would take an output char* parameter instead of a FILE*,
  2. a simpler API to just get the human readable message (if any) for the most likely "root cause" for the latest error.

1 because if you're interested in getting the error trace into a string (for e.g. showing it in the UI), then setting up an in-memory FILE* is not something that can be easily done in a portable way (I know about fmemopen/open_memstream, but they are POSIX). I know this can be done by setting up your own error stack walking function, but it's quite awkward to have to do that, especially if you're happy with the formatting that HDF5 does by default, and you just want the result as a string instead.

2 because like others have mentioned, sometimes you're not interested in the full trace, but only the likely root cause (which is probably somewhere at the bottom of the stack).

Anyway, thanks for chiming in, and sorry again for not following up on my own post earlier.

Elvis
 

HDF5-DIAG: Error detected in HDF5 (1.8.16) thread 0:
  #000: ../../../hdf5-1.8.16/src/H5F.c line 604 in H5Fopen(): unable to open file
    major: File accessibilty
    minor: Unable to open file
  #001: ../../../hdf5-1.8.16/src/H5Fint.c line 992 in H5F_open(): unable to open file: time = Mon Nov 28 14:41:40 2016
, name = 'try.h5', tent_flags = 1
    major: File accessibilty
    minor: Unable to open file
  #002: ../../../hdf5-1.8.16/src/H5FD.c line 993 in H5FD_open(): open failed
    major: Virtual File Layer
    minor: Unable to initialize object
  #003: ../../../hdf5-1.8.16/src/H5FDsec2.c line 339 in H5FD_sec2_open(): unable to open file: name = 'try.h5', errno = 13, error message = 'Permission denied', flags = 1, o_flags = 2
    major: File accessibilty
    minor: Unable to open file


Thanks in advance,
Elvis

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: More informative error from H5Fopen?

Elvis Stansvik
In reply to this post by Landon Clipp
2016-11-28 23:28 GMT+01:00 Landon Clipp <[hidden email]>:

Also one more possible issue I'd add onto my list there: it is possible that the file is corrupt. So try ruling that out by making an empty HDF5 file in the same directory as the one you're trying to access and run the same functions, only opening this new file instead.

Ah yes, sorry I should have been more clear: In my case I wasn't debugging a particular failure, just looking for a way to get as much info out of HDF5 as possible to present together with an error message to the user (it's a GUI application) (see also my response to Rob just now).

Thanks for this suggestion and the ones below as well though. They will surely help when things truly hit the fan :)

Elvis

Regards,
Landon


On Nov 28, 2016 4:26 PM, "Landon Clipp" <[hidden email]> wrote:

Mark,

I get what you mean. I have rarely found the error output of HDF5 to be of much use, although sometimes it can be helpful. For instance just off the top of my head, I've been getting errors about HDF attributes. I've called H5Aclose and I get errors about the argument not being an attribute. Turns out the variable I passed was never initialized. So in that case it's helpful.

One thing to realize is that the error is most likely NOT reading in the library itself. I recommend that you use a debugger like GDB to find exactly what you are passing into the function and to see if every value you're passing in makes sense. That's really the only way to go about it.

It all comes down to this: H5Fopen is failing and there are a finite number of issues we can address.

1. Does the file exist, and if so are you providing the correct path to it (relative or absolute)? Do you have read/write permission for the file?
2. Have you misspelled the file?
3. Are you providing either DF_ACC_RDWR or DF_ACC_RDONLY to the second argument? Only those two arguments are valid.
4. What file access property list are you providing? You should be able to rule out providing a buggy access property by just passing in the default parameter.

I guarantee that your error lies in one of these issues. I recommend you make the second and third arguments the read only and default respectively. That will isolate the problem to your first argument and whether or not you have appropriate file permissions.

Regards,
Landon


On Nov 28, 2016 3:33 PM, "Miller, Mark C." <[hidden email]> wrote:
"Hdf-forum on behalf of Rob Latham" wrote:


I think you are going to have to share your output with the list.
Whenever *I* run hdf5 and get an error I get screenfuls of information.

For example, here's a simple hdf5 program trying to open a file for
which I have removed all read and write permissions:

HDF5-DIAG: Error detected in HDF5 (1.8.16) thread 0:
   #000: ../../../hdf5-1.8.16/src/H5F.c line 604 in H5Fopen(): unable to
open file
     major: File accessibilty
     minor: Unable to open file
   #001: ../../../hdf5-1.8.16/src/H5Fint.c line 992 in H5F_open():
unable to open file: time = Mon Nov 28 14:41:40 2016
, name = 'try.h5', tent_flags = 1
     major: File accessibilty
     minor: Unable to open file
   #002: ../../../hdf5-1.8.16/src/H5FD.c line 993 in H5FD_open(): open
failed
     major: Virtual File Layer
     minor: Unable to initialize object
   #003: ../../../hdf5-1.8.16/src/H5FDsec2.c line 339 in
H5FD_sec2_open(): unable to open file: name = 'try.h5', errno = 13,
error message = 'Permission denied', flags = 1, o_flags = 2
     major: File accessibilty
     minor: Unable to open file


I am not sure how other users feel about this and it may not even be the original poster's ( Elvis Stansvik) complaint but a big challenge I have with interpreting HDF5 errors *is* the "...screenfuls of information".

It is challenging to paw through it to discover the root cause.

For example, in the above error stack, we get information about why sec2 VFD failed..."errno=13 ("Permission enied") but what are 'flags' and 'o_flags' there? Are these params to sec2 open call? That has only an 'oflag' argument and not an additional 'flags' argument. And, their numeric values, '1' and '2' are too informatvie either. What about similar info at the HDF5 public interface (e.g. H5Fopen)? What was passed for 'flags' there. Oh, maybe 'flags' in the #003 trace *is* the 'flags' argument caller passed to H5Fopen and 'o_flags' is what HDF5 lib passed to sec2 open call? I could see having 'flags' emitted in the #000 trace if that is indeed the argument passed as 'flags' to H5Fopen. and 'oflags' not 'o_flags' emitted in the #003 trace. Finally, it would be *much* more helpful to see the symbolic constants for the flags and not their numerics.

So, I personally find grocking HDF5 error stack output very awkward. 

It may be that it is awkward enough for others that no one really is able to find the information they needed in it (e.g. the signal to noise ratio is just way too low).

My two cents.

Mark

-- 
Mark C. Miller, LLNL

"In the end, we will remember not the words of
our enemies but the silence of our friends" - MLK

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: More informative error from H5Fopen?

Allen Byrne
In reply to this post by Elvis Stansvik
If you download the latest source, with the Java API, or the hdf-java 3.2.1 source - you can inspect the JNI code for exceptions in the native hdf5 exceptionImp.c file.
Another source of using the exception API for custom error reporting is the tools library and h5dump.

Allen

On Tuesday, November 29, 2016 9:20:03 AM CST Elvis Stansvik wrote:

> Hi Rob,
>
> 2016-11-28 21:43 GMT+01:00 Rob Latham <[hidden email]>:
>
> >
> >
> > On 11/08/2016 01:18 PM, Elvis Stansvik wrote:
> >
> >> Hi all,
> >>
> >> Is it possible to get some more informative error from H5Fopen than
> >> "it failed"? E.g. if it was a permission problem, nonexistent file,
> >> corrupted file, ...? Or must I try to deduce the reason for the
> >> failure myself? (always risky to do such checks pre- or post-mortem so
> >> to speak..).
> >>
> >
> > I think you are going to have to share your output with the list. Whenever
> > *I* run hdf5 and get an error I get screenfuls of information.
> >
> > For example, here's a simple hdf5 program trying to open a file for which
> > I have removed all read and write permissions:
> >
>
> Yes, and I'm sorry. I should have replied to myself quite soon after I sent
> my message, but somehow forgot about it.
>
> When I first wrote my message, I was
>
>   1. working through the C++ API, and
>   2. building my program in release mode.
>
> Working through the C++ API meant I was using
> H5::Exception::getDetailMsg(), hoping to get some detailed message, but it
> was giving me something like "H5Fopen failed" (which is why I mentioned the
> H5Fopen C function in my post). Building my program in release mode also
> meant I did not get the detailed error stack printout that the C layer does
> by default in debug mode (like you showed below). That's why I was a little
> puzzled.
>
> Since I wrote my message, I have
>
>   1. switched to working with the C API (through a simple HDF5File C++
> class of my own),
>   2. learned all about the gory details of the H5E API [1] and error
> handling in chapter 9 of the User Guide.
>
> So I'm all good now :) Though I must say the error handling API is quite
> complicated (I still haven't written my custom error walking/formatting
> functions, so only relying on the default printouts I get in debug mode).
>
> One thing that I would wish for is
>
>   1. an API similar to H5Eprint that would take an output char* parameter
> instead of a FILE*,
>   2. a simpler API to just get the human readable message (if any) for the
> most likely "root cause" for the latest error.
>
> 1 because if you're interested in getting the error trace into a string
> (for e.g. showing it in the UI), then setting up an in-memory FILE* is not
> something that can be easily done in a portable way (I know about
> fmemopen/open_memstream, but they are POSIX). I know this can be done by
> setting up your own error stack walking function, but it's quite awkward to
> have to do that, especially if you're happy with the formatting that HDF5
> does by default, and you just want the result as a string instead.
>
> 2 because like others have mentioned, sometimes you're not interested in
> the full trace, but only the likely root cause (which is probably somewhere
> at the bottom of the stack).
>
> Anyway, thanks for chiming in, and sorry again for not following up on my
> own post earlier.
>
> Elvis
>
>
> >
> > HDF5-DIAG: Error detected in HDF5 (1.8.16) thread 0:
> >   #000: ../../../hdf5-1.8.16/src/H5F.c line 604 in H5Fopen(): unable to
> > open file
> >     major: File accessibilty
> >     minor: Unable to open file
> >   #001: ../../../hdf5-1.8.16/src/H5Fint.c line 992 in H5F_open(): unable
> > to open file: time = Mon Nov 28 14:41:40 2016
> > , name = 'try.h5', tent_flags = 1
> >     major: File accessibilty
> >     minor: Unable to open file
> >   #002: ../../../hdf5-1.8.16/src/H5FD.c line 993 in H5FD_open(): open
> > failed
> >     major: Virtual File Layer
> >     minor: Unable to initialize object
> >   #003: ../../../hdf5-1.8.16/src/H5FDsec2.c line 339 in H5FD_sec2_open():
> > unable to open file: name = 'try.h5', errno = 13, error message =
> > 'Permission denied', flags = 1, o_flags = 2
> >     major: File accessibilty
> >     minor: Unable to open file
> >
> >
> >> Thanks in advance,
> >> Elvis
> >>
> >> _______________________________________________
> >> Hdf-forum is for HDF software users discussion.
> >> [hidden email]
> >> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> >> Twitter: https://twitter.com/hdf5
> >>
> >>
> > _______________________________________________
> > Hdf-forum is for HDF software users discussion.
> > [hidden email]
> > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> > Twitter: https://twitter.com/hdf5
> >
>



_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: More informative error from H5Fopen?

Elvis Stansvik

Den 29 nov. 2016 3:32 em skrev "Allen Byrne" <[hidden email]>:
>
> If you download the latest source, with the Java API, or the hdf-java 3.2.1 source - you can inspect the JNI code for exceptions in the native hdf5 exceptionImp.c file.
> Another source of using the exception API for custom error reporting is the tools library and h5dump.

Ah, thanks for the pointers Allen. I actually think I came across the JNI code during my googlings, but didn't look closer at it before I got busy with other things. I'll take a look when I'm back at it.

Elvis

>
> Allen
>
> On Tuesday, November 29, 2016 9:20:03 AM CST Elvis Stansvik wrote:
> > Hi Rob,
> >
> > 2016-11-28 21:43 GMT+01:00 Rob Latham <[hidden email]>:
> >
> > >
> > >
> > > On 11/08/2016 01:18 PM, Elvis Stansvik wrote:
> > >
> > >> Hi all,
> > >>
> > >> Is it possible to get some more informative error from H5Fopen than
> > >> "it failed"? E.g. if it was a permission problem, nonexistent file,
> > >> corrupted file, ...? Or must I try to deduce the reason for the
> > >> failure myself? (always risky to do such checks pre- or post-mortem so
> > >> to speak..).
> > >>
> > >
> > > I think you are going to have to share your output with the list. Whenever
> > > *I* run hdf5 and get an error I get screenfuls of information.
> > >
> > > For example, here's a simple hdf5 program trying to open a file for which
> > > I have removed all read and write permissions:
> > >
> >
> > Yes, and I'm sorry. I should have replied to myself quite soon after I sent
> > my message, but somehow forgot about it.
> >
> > When I first wrote my message, I was
> >
> >   1. working through the C++ API, and
> >   2. building my program in release mode.
> >
> > Working through the C++ API meant I was using
> > H5::Exception::getDetailMsg(), hoping to get some detailed message, but it
> > was giving me something like "H5Fopen failed" (which is why I mentioned the
> > H5Fopen C function in my post). Building my program in release mode also
> > meant I did not get the detailed error stack printout that the C layer does
> > by default in debug mode (like you showed below). That's why I was a little
> > puzzled.
> >
> > Since I wrote my message, I have
> >
> >   1. switched to working with the C API (through a simple HDF5File C++
> > class of my own),
> >   2. learned all about the gory details of the H5E API [1] and error
> > handling in chapter 9 of the User Guide.
> >
> > So I'm all good now :) Though I must say the error handling API is quite
> > complicated (I still haven't written my custom error walking/formatting
> > functions, so only relying on the default printouts I get in debug mode).
> >
> > One thing that I would wish for is
> >
> >   1. an API similar to H5Eprint that would take an output char* parameter
> > instead of a FILE*,
> >   2. a simpler API to just get the human readable message (if any) for the
> > most likely "root cause" for the latest error.
> >
> > 1 because if you're interested in getting the error trace into a string
> > (for e.g. showing it in the UI), then setting up an in-memory FILE* is not
> > something that can be easily done in a portable way (I know about
> > fmemopen/open_memstream, but they are POSIX). I know this can be done by
> > setting up your own error stack walking function, but it's quite awkward to
> > have to do that, especially if you're happy with the formatting that HDF5
> > does by default, and you just want the result as a string instead.
> >
> > 2 because like others have mentioned, sometimes you're not interested in
> > the full trace, but only the likely root cause (which is probably somewhere
> > at the bottom of the stack).
> >
> > Anyway, thanks for chiming in, and sorry again for not following up on my
> > own post earlier.
> >
> > Elvis
> >
> >
> > >
> > > HDF5-DIAG: Error detected in HDF5 (1.8.16) thread 0:
> > >   #000: ../../../hdf5-1.8.16/src/H5F.c line 604 in H5Fopen(): unable to
> > > open file
> > >     major: File accessibilty
> > >     minor: Unable to open file
> > >   #001: ../../../hdf5-1.8.16/src/H5Fint.c line 992 in H5F_open(): unable
> > > to open file: time = Mon Nov 28 14:41:40 2016
> > > , name = 'try.h5', tent_flags = 1
> > >     major: File accessibilty
> > >     minor: Unable to open file
> > >   #002: ../../../hdf5-1.8.16/src/H5FD.c line 993 in H5FD_open(): open
> > > failed
> > >     major: Virtual File Layer
> > >     minor: Unable to initialize object
> > >   #003: ../../../hdf5-1.8.16/src/H5FDsec2.c line 339 in H5FD_sec2_open():
> > > unable to open file: name = 'try.h5', errno = 13, error message =
> > > 'Permission denied', flags = 1, o_flags = 2
> > >     major: File accessibilty
> > >     minor: Unable to open file
> > >
> > >
> > >> Thanks in advance,
> > >> Elvis
> > >>
> > >> _______________________________________________
> > >> Hdf-forum is for HDF software users discussion.
> > >> [hidden email]
> > >> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> > >> Twitter: https://twitter.com/hdf5
> > >>
> > >>
> > > _______________________________________________
> > > Hdf-forum is for HDF software users discussion.
> > > [hidden email]
> > > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> > > Twitter: https://twitter.com/hdf5
> > >
> >
>
>


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: More informative error from H5Fopen?

Elvis Stansvik

Den 29 nov. 2016 5:41 em skrev "Elvis Stansvik" <[hidden email]>:
>
> Den 29 nov. 2016 3:32 em skrev "Allen Byrne" <[hidden email]>:
> >
> > If you download the latest source, with the Java API, or the hdf-java 3.2.1 source - you can inspect the JNI code for exceptions in the native hdf5 exceptionImp.c file.
> > Another source of using the exception API for custom error reporting is the tools library and h5dump.
>
> Ah, thanks for the pointers Allen. I actually think I came across the JNI code during my googlings, but didn't look closer at it before I got busy with other things. I'll take a look when I'm back at it.

I looked at it briefly again, and the JNI code looks like it's doing exactly what I want (a custom walker + H5E_walk2 to just get the top error, and then H5E_get_msg to get the corresponding message). Many thanks.

I think when I was trying to figure it out on my own, I didn't realize it's the minor error code I should feed to H5E_get_msg.

Elvis

> Elvis
>
> >
> > Allen
> >
> > On Tuesday, November 29, 2016 9:20:03 AM CST Elvis Stansvik wrote:
> > > Hi Rob,
> > >
> > > 2016-11-28 21:43 GMT+01:00 Rob Latham <[hidden email]>:
> > >
> > > >
> > > >
> > > > On 11/08/2016 01:18 PM, Elvis Stansvik wrote:
> > > >
> > > >> Hi all,
> > > >>
> > > >> Is it possible to get some more informative error from H5Fopen than
> > > >> "it failed"? E.g. if it was a permission problem, nonexistent file,
> > > >> corrupted file, ...? Or must I try to deduce the reason for the
> > > >> failure myself? (always risky to do such checks pre- or post-mortem so
> > > >> to speak..).
> > > >>
> > > >
> > > > I think you are going to have to share your output with the list. Whenever
> > > > *I* run hdf5 and get an error I get screenfuls of information.
> > > >
> > > > For example, here's a simple hdf5 program trying to open a file for which
> > > > I have removed all read and write permissions:
> > > >
> > >
> > > Yes, and I'm sorry. I should have replied to myself quite soon after I sent
> > > my message, but somehow forgot about it.
> > >
> > > When I first wrote my message, I was
> > >
> > >   1. working through the C++ API, and
> > >   2. building my program in release mode.
> > >
> > > Working through the C++ API meant I was using
> > > H5::Exception::getDetailMsg(), hoping to get some detailed message, but it
> > > was giving me something like "H5Fopen failed" (which is why I mentioned the
> > > H5Fopen C function in my post). Building my program in release mode also
> > > meant I did not get the detailed error stack printout that the C layer does
> > > by default in debug mode (like you showed below). That's why I was a little
> > > puzzled.
> > >
> > > Since I wrote my message, I have
> > >
> > >   1. switched to working with the C API (through a simple HDF5File C++
> > > class of my own),
> > >   2. learned all about the gory details of the H5E API [1] and error
> > > handling in chapter 9 of the User Guide.
> > >
> > > So I'm all good now :) Though I must say the error handling API is quite
> > > complicated (I still haven't written my custom error walking/formatting
> > > functions, so only relying on the default printouts I get in debug mode).
> > >
> > > One thing that I would wish for is
> > >
> > >   1. an API similar to H5Eprint that would take an output char* parameter
> > > instead of a FILE*,
> > >   2. a simpler API to just get the human readable message (if any) for the
> > > most likely "root cause" for the latest error.
> > >
> > > 1 because if you're interested in getting the error trace into a string
> > > (for e.g. showing it in the UI), then setting up an in-memory FILE* is not
> > > something that can be easily done in a portable way (I know about
> > > fmemopen/open_memstream, but they are POSIX). I know this can be done by
> > > setting up your own error stack walking function, but it's quite awkward to
> > > have to do that, especially if you're happy with the formatting that HDF5
> > > does by default, and you just want the result as a string instead.
> > >
> > > 2 because like others have mentioned, sometimes you're not interested in
> > > the full trace, but only the likely root cause (which is probably somewhere
> > > at the bottom of the stack).
> > >
> > > Anyway, thanks for chiming in, and sorry again for not following up on my
> > > own post earlier.
> > >
> > > Elvis
> > >
> > >
> > > >
> > > > HDF5-DIAG: Error detected in HDF5 (1.8.16) thread 0:
> > > >   #000: ../../../hdf5-1.8.16/src/H5F.c line 604 in H5Fopen(): unable to
> > > > open file
> > > >     major: File accessibilty
> > > >     minor: Unable to open file
> > > >   #001: ../../../hdf5-1.8.16/src/H5Fint.c line 992 in H5F_open(): unable
> > > > to open file: time = Mon Nov 28 14:41:40 2016
> > > > , name = 'try.h5', tent_flags = 1
> > > >     major: File accessibilty
> > > >     minor: Unable to open file
> > > >   #002: ../../../hdf5-1.8.16/src/H5FD.c line 993 in H5FD_open(): open
> > > > failed
> > > >     major: Virtual File Layer
> > > >     minor: Unable to initialize object
> > > >   #003: ../../../hdf5-1.8.16/src/H5FDsec2.c line 339 in H5FD_sec2_open():
> > > > unable to open file: name = 'try.h5', errno = 13, error message =
> > > > 'Permission denied', flags = 1, o_flags = 2
> > > >     major: File accessibilty
> > > >     minor: Unable to open file
> > > >
> > > >
> > > >> Thanks in advance,
> > > >> Elvis
> > > >>
> > > >> _______________________________________________
> > > >> Hdf-forum is for HDF software users discussion.
> > > >> [hidden email]
> > > >> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> > > >> Twitter: https://twitter.com/hdf5
> > > >>
> > > >>
> > > > _______________________________________________
> > > > Hdf-forum is for HDF software users discussion.
> > > > [hidden email]
> > > > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> > > > Twitter: https://twitter.com/hdf5
> > > >
> > >
> >
> >


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: More informative error from H5Fopen?

Allen Byrne
One thing to keep in mind, and the JNI code does, is that you clear the error stack when you use certain H5E APIs.

Allen

On Tuesday, November 29, 2016 6:07:22 PM CST Elvis Stansvik wrote:

> Den 29 nov. 2016 5:41 em skrev "Elvis Stansvik" <
> [hidden email]>:
> >
> > Den 29 nov. 2016 3:32 em skrev "Allen Byrne" <[hidden email]>:
> > >
> > > If you download the latest source, with the Java API, or the hdf-java
> 3.2.1 source - you can inspect the JNI code for exceptions in the native
> hdf5 exceptionImp.c file.
> > > Another source of using the exception API for custom error reporting is
> the tools library and h5dump.
> >
> > Ah, thanks for the pointers Allen. I actually think I came across the JNI
> code during my googlings, but didn't look closer at it before I got busy
> with other things. I'll take a look when I'm back at it.
>
> I looked at it briefly again, and the JNI code looks like it's doing
> exactly what I want (a custom walker + H5E_walk2 to just get the top error,
> and then H5E_get_msg to get the corresponding message). Many thanks.
>
> I think when I was trying to figure it out on my own, I didn't realize it's
> the minor error code I should feed to H5E_get_msg.
>
> Elvis
>
> > Elvis
> >
> > >
> > > Allen
> > >
> > > On Tuesday, November 29, 2016 9:20:03 AM CST Elvis Stansvik wrote:
> > > > Hi Rob,
> > > >
> > > > 2016-11-28 21:43 GMT+01:00 Rob Latham <[hidden email]>:
> > > >
> > > > >
> > > > >
> > > > > On 11/08/2016 01:18 PM, Elvis Stansvik wrote:
> > > > >
> > > > >> Hi all,
> > > > >>
> > > > >> Is it possible to get some more informative error from H5Fopen than
> > > > >> "it failed"? E.g. if it was a permission problem, nonexistent file,
> > > > >> corrupted file, ...? Or must I try to deduce the reason for the
> > > > >> failure myself? (always risky to do such checks pre- or
> post-mortem so
> > > > >> to speak..).
> > > > >>
> > > > >
> > > > > I think you are going to have to share your output with the list.
> Whenever
> > > > > *I* run hdf5 and get an error I get screenfuls of information.
> > > > >
> > > > > For example, here's a simple hdf5 program trying to open a file for
> which
> > > > > I have removed all read and write permissions:
> > > > >
> > > >
> > > > Yes, and I'm sorry. I should have replied to myself quite soon after
> I sent
> > > > my message, but somehow forgot about it.
> > > >
> > > > When I first wrote my message, I was
> > > >
> > > >   1. working through the C++ API, and
> > > >   2. building my program in release mode.
> > > >
> > > > Working through the C++ API meant I was using
> > > > H5::Exception::getDetailMsg(), hoping to get some detailed message,
> but it
> > > > was giving me something like "H5Fopen failed" (which is why I
> mentioned the
> > > > H5Fopen C function in my post). Building my program in release mode
> also
> > > > meant I did not get the detailed error stack printout that the C
> layer does
> > > > by default in debug mode (like you showed below). That's why I was a
> little
> > > > puzzled.
> > > >
> > > > Since I wrote my message, I have
> > > >
> > > >   1. switched to working with the C API (through a simple HDF5File C++
> > > > class of my own),
> > > >   2. learned all about the gory details of the H5E API [1] and error
> > > > handling in chapter 9 of the User Guide.
> > > >
> > > > So I'm all good now :) Though I must say the error handling API is
> quite
> > > > complicated (I still haven't written my custom error
> walking/formatting
> > > > functions, so only relying on the default printouts I get in debug
> mode).
> > > >
> > > > One thing that I would wish for is
> > > >
> > > >   1. an API similar to H5Eprint that would take an output char*
> parameter
> > > > instead of a FILE*,
> > > >   2. a simpler API to just get the human readable message (if any)
> for the
> > > > most likely "root cause" for the latest error.
> > > >
> > > > 1 because if you're interested in getting the error trace into a
> string
> > > > (for e.g. showing it in the UI), then setting up an in-memory FILE*
> is not
> > > > something that can be easily done in a portable way (I know about
> > > > fmemopen/open_memstream, but they are POSIX). I know this can be done
> by
> > > > setting up your own error stack walking function, but it's quite
> awkward to
> > > > have to do that, especially if you're happy with the formatting that
> HDF5
> > > > does by default, and you just want the result as a string instead.
> > > >
> > > > 2 because like others have mentioned, sometimes you're not interested
> in
> > > > the full trace, but only the likely root cause (which is probably
> somewhere
> > > > at the bottom of the stack).
> > > >
> > > > Anyway, thanks for chiming in, and sorry again for not following up
> on my
> > > > own post earlier.
> > > >
> > > > Elvis
> > > >
> > > >
> > > > >
> > > > > HDF5-DIAG: Error detected in HDF5 (1.8.16) thread 0:
> > > > >   #000: ../../../hdf5-1.8.16/src/H5F.c line 604 in H5Fopen():
> unable to
> > > > > open file
> > > > >     major: File accessibilty
> > > > >     minor: Unable to open file
> > > > >   #001: ../../../hdf5-1.8.16/src/H5Fint.c line 992 in H5F_open():
> unable
> > > > > to open file: time = Mon Nov 28 14:41:40 2016
> > > > > , name = 'try.h5', tent_flags = 1
> > > > >     major: File accessibilty
> > > > >     minor: Unable to open file
> > > > >   #002: ../../../hdf5-1.8.16/src/H5FD.c line 993 in H5FD_open():
> open
> > > > > failed
> > > > >     major: Virtual File Layer
> > > > >     minor: Unable to initialize object
> > > > >   #003: ../../../hdf5-1.8.16/src/H5FDsec2.c line 339 in
> H5FD_sec2_open():
> > > > > unable to open file: name = 'try.h5', errno = 13, error message =
> > > > > 'Permission denied', flags = 1, o_flags = 2
> > > > >     major: File accessibilty
> > > > >     minor: Unable to open file
> > > > >
> > > > >
> > > > >> Thanks in advance,
> > > > >> Elvis
> > > > >>
> > > > >> _______________________________________________
> > > > >> Hdf-forum is for HDF software users discussion.
> > > > >> [hidden email]
> > > > >>
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> > > > >> Twitter: https://twitter.com/hdf5
> > > > >>
> > > > >>
> > > > > _______________________________________________
> > > > > Hdf-forum is for HDF software users discussion.
> > > > > [hidden email]
> > > > >
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> > > > > Twitter: https://twitter.com/hdf5
> > > > >
> > > >
> > >
> > >
>



_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: More informative error from H5Fopen?

Elvis Stansvik

Den 29 nov. 2016 10:45 em skrev "Allen Byrne" <[hidden email]>:
>
> One thing to keep in mind, and the JNI code does, is that you clear the error stack when you use certain H5E APIs.

Thanks for the heads up. Will proceed with care.

Elvis
>
> Allen
>
> On Tuesday, November 29, 2016 6:07:22 PM CST Elvis Stansvik wrote:
> > Den 29 nov. 2016 5:41 em skrev "Elvis Stansvik" <
> > [hidden email]>:
> > >
> > > Den 29 nov. 2016 3:32 em skrev "Allen Byrne" <[hidden email]>:
> > > >
> > > > If you download the latest source, with the Java API, or the hdf-java
> > 3.2.1 source - you can inspect the JNI code for exceptions in the native
> > hdf5 exceptionImp.c file.
> > > > Another source of using the exception API for custom error reporting is
> > the tools library and h5dump.
> > >
> > > Ah, thanks for the pointers Allen. I actually think I came across the JNI
> > code during my googlings, but didn't look closer at it before I got busy
> > with other things. I'll take a look when I'm back at it.
> >
> > I looked at it briefly again, and the JNI code looks like it's doing
> > exactly what I want (a custom walker + H5E_walk2 to just get the top error,
> > and then H5E_get_msg to get the corresponding message). Many thanks.
> >
> > I think when I was trying to figure it out on my own, I didn't realize it's
> > the minor error code I should feed to H5E_get_msg.
> >
> > Elvis
> >
> > > Elvis
> > >
> > > >
> > > > Allen
> > > >
> > > > On Tuesday, November 29, 2016 9:20:03 AM CST Elvis Stansvik wrote:
> > > > > Hi Rob,
> > > > >
> > > > > 2016-11-28 21:43 GMT+01:00 Rob Latham <[hidden email]>:
> > > > >
> > > > > >
> > > > > >
> > > > > > On 11/08/2016 01:18 PM, Elvis Stansvik wrote:
> > > > > >
> > > > > >> Hi all,
> > > > > >>
> > > > > >> Is it possible to get some more informative error from H5Fopen than
> > > > > >> "it failed"? E.g. if it was a permission problem, nonexistent file,
> > > > > >> corrupted file, ...? Or must I try to deduce the reason for the
> > > > > >> failure myself? (always risky to do such checks pre- or
> > post-mortem so
> > > > > >> to speak..).
> > > > > >>
> > > > > >
> > > > > > I think you are going to have to share your output with the list.
> > Whenever
> > > > > > *I* run hdf5 and get an error I get screenfuls of information.
> > > > > >
> > > > > > For example, here's a simple hdf5 program trying to open a file for
> > which
> > > > > > I have removed all read and write permissions:
> > > > > >
> > > > >
> > > > > Yes, and I'm sorry. I should have replied to myself quite soon after
> > I sent
> > > > > my message, but somehow forgot about it.
> > > > >
> > > > > When I first wrote my message, I was
> > > > >
> > > > >   1. working through the C++ API, and
> > > > >   2. building my program in release mode.
> > > > >
> > > > > Working through the C++ API meant I was using
> > > > > H5::Exception::getDetailMsg(), hoping to get some detailed message,
> > but it
> > > > > was giving me something like "H5Fopen failed" (which is why I
> > mentioned the
> > > > > H5Fopen C function in my post). Building my program in release mode
> > also
> > > > > meant I did not get the detailed error stack printout that the C
> > layer does
> > > > > by default in debug mode (like you showed below). That's why I was a
> > little
> > > > > puzzled.
> > > > >
> > > > > Since I wrote my message, I have
> > > > >
> > > > >   1. switched to working with the C API (through a simple HDF5File C++
> > > > > class of my own),
> > > > >   2. learned all about the gory details of the H5E API [1] and error
> > > > > handling in chapter 9 of the User Guide.
> > > > >
> > > > > So I'm all good now :) Though I must say the error handling API is
> > quite
> > > > > complicated (I still haven't written my custom error
> > walking/formatting
> > > > > functions, so only relying on the default printouts I get in debug
> > mode).
> > > > >
> > > > > One thing that I would wish for is
> > > > >
> > > > >   1. an API similar to H5Eprint that would take an output char*
> > parameter
> > > > > instead of a FILE*,
> > > > >   2. a simpler API to just get the human readable message (if any)
> > for the
> > > > > most likely "root cause" for the latest error.
> > > > >
> > > > > 1 because if you're interested in getting the error trace into a
> > string
> > > > > (for e.g. showing it in the UI), then setting up an in-memory FILE*
> > is not
> > > > > something that can be easily done in a portable way (I know about
> > > > > fmemopen/open_memstream, but they are POSIX). I know this can be done
> > by
> > > > > setting up your own error stack walking function, but it's quite
> > awkward to
> > > > > have to do that, especially if you're happy with the formatting that
> > HDF5
> > > > > does by default, and you just want the result as a string instead.
> > > > >
> > > > > 2 because like others have mentioned, sometimes you're not interested
> > in
> > > > > the full trace, but only the likely root cause (which is probably
> > somewhere
> > > > > at the bottom of the stack).
> > > > >
> > > > > Anyway, thanks for chiming in, and sorry again for not following up
> > on my
> > > > > own post earlier.
> > > > >
> > > > > Elvis
> > > > >
> > > > >
> > > > > >
> > > > > > HDF5-DIAG: Error detected in HDF5 (1.8.16) thread 0:
> > > > > >   #000: ../../../hdf5-1.8.16/src/H5F.c line 604 in H5Fopen():
> > unable to
> > > > > > open file
> > > > > >     major: File accessibilty
> > > > > >     minor: Unable to open file
> > > > > >   #001: ../../../hdf5-1.8.16/src/H5Fint.c line 992 in H5F_open():
> > unable
> > > > > > to open file: time = Mon Nov 28 14:41:40 2016
> > > > > > , name = 'try.h5', tent_flags = 1
> > > > > >     major: File accessibilty
> > > > > >     minor: Unable to open file
> > > > > >   #002: ../../../hdf5-1.8.16/src/H5FD.c line 993 in H5FD_open():
> > open
> > > > > > failed
> > > > > >     major: Virtual File Layer
> > > > > >     minor: Unable to initialize object
> > > > > >   #003: ../../../hdf5-1.8.16/src/H5FDsec2.c line 339 in
> > H5FD_sec2_open():
> > > > > > unable to open file: name = 'try.h5', errno = 13, error message =
> > > > > > 'Permission denied', flags = 1, o_flags = 2
> > > > > >     major: File accessibilty
> > > > > >     minor: Unable to open file
> > > > > >
> > > > > >
> > > > > >> Thanks in advance,
> > > > > >> Elvis
> > > > > >>
> > > > > >> _______________________________________________
> > > > > >> Hdf-forum is for HDF software users discussion.
> > > > > >> [hidden email]
> > > > > >>
> > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> > > > > >> Twitter: https://twitter.com/hdf5
> > > > > >>
> > > > > >>
> > > > > > _______________________________________________
> > > > > > Hdf-forum is for HDF software users discussion.
> > > > > > [hidden email]
> > > > > >
> > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> > > > > > Twitter: https://twitter.com/hdf5
> > > > > >
> > > > >
> > > >
> > > >
> >
>
>


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Loading...