Quantcast

Memory leak in H5LTopen_file_image

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Memory leak in H5LTopen_file_image

Samer Afach-4
Dear HDF5 experts:

I've been investigating memory leaks in a program of mine, and then found that the function H5LTopen_file_image is causing the leak. I created a minimal example reproducing the problem, which just opens and closes a file again and again and again, after reading the file into memory only once, and one could see a clear memory build-up. Could someone please verify that I'm not doing anything wrong?

Below is the minimal working example of the program. I tested it with valgrind/massif, using the command:

$ valgrind --tool=massif ./LeakTest

And one could obviously see the memory rise. What I have in my application is a file in memory (as in the example) that I need to read without modifying it or releasing it. The only way I was able to get rid of the memory leak, is by using the flag H5LT_FILE_IMAGE_OPEN_RW, which means that the image will be copied, and I'm trying to avoid that. Is this a bug in the HDF5 library? Or am I doing something wrong?

#include <iostream>
#include <fstream>
#include <hdf5.h>
#include <hdf5_hl.h>

    
std::size_t GetFileSize(const std::string &filename)
{
    std::ifstream file(filename.c_str(), std::ios::binary | std::ios::ate);
    return static_cast<size_t>(file.tellg());
}

    
void ReadBinFileToString(const std::string &filename, std::string &data)
{
    std::fstream fileObject(filename.c_str(),std::ios::in | std::ios::binary);
    size_t filesize = GetFileSize(filename);
    data.resize(filesize);
    if(filesize > 0) {
        fileObject.read(&data.front(),filesize);
        fileObject.close();
    }
    else {
        std::cout<<"Error reading file"<<std::endl;
        std::exit(1);
    }
}

    
int main()
{
    std::string h5data;
    ReadBinFileToString("../Test.h5",h5data);

    
    long numOfTests = 25000;
    for(long i = 0; i < numOfTests; i++)
    {
        hid_t fileHandle = H5LTopen_file_image((void*)&h5data.front(),h5data.size(),H5LT_FILE_IMAGE_DONT_COPY | H5LT_FILE_IMAGE_DONT_RELEASE);
        H5Fclose(fileHandle);
        std::cout<<"\r";std::cout<< i << ":" <<  numOfTests;std::cout.flush();
    }
    return 0;
}

Thank you.

Best,
Samer Afach

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Memory leak in H5LTopen_file_image

Samer Afach-4

Thank you for approving of this post. It was sent a few days ago, and is posted with an old date. So I'm replying to bump it up.


On 29.10.2016 4:13 AM, Samer Afach wrote:
Dear HDF5 experts:

I've been investigating memory leaks in a program of mine, and then found that the function H5LTopen_file_image is causing the leak. I created a minimal example reproducing the problem, which just opens and closes a file again and again and again, after reading the file into memory only once, and one could see a clear memory build-up. Could someone please verify that I'm not doing anything wrong?

Below is the minimal working example of the program. I tested it with valgrind/massif, using the command:

$ valgrind --tool=massif ./LeakTest

And one could obviously see the memory rise. What I have in my application is a file in memory (as in the example) that I need to read without modifying it or releasing it. The only way I was able to get rid of the memory leak, is by using the flag H5LT_FILE_IMAGE_OPEN_RW, which means that the image will be copied, and I'm trying to avoid that. Is this a bug in the HDF5 library? Or am I doing something wrong?

#include <iostream>
#include <fstream>
#include <hdf5.h>
#include <hdf5_hl.h>
std::size_t GetFileSize(const std::string &filename)
{
    std::ifstream file(filename.c_str(), std::ios::binary | std::ios::ate);
    return static_cast<size_t>(file.tellg());
}
void ReadBinFileToString(const std::string &filename, std::string &data)
{
    std::fstream fileObject(filename.c_str(),std::ios::in | std::ios::binary);
    size_t filesize = GetFileSize(filename);
    data.resize(filesize);
    if(filesize > 0) {
        fileObject.read(&data.front(),filesize);
        fileObject.close();
    }
    else {
        std::cout<<"Error reading file"<<std::endl;
        std::exit(1);
    }
}
int main()
{
    std::string h5data;
    ReadBinFileToString("../Test.h5",h5data);
    long numOfTests = 25000;
    for(long i = 0; i < numOfTests; i++)
    {
        hid_t fileHandle = H5LTopen_file_image((void*)&h5data.front(),h5data.size(),H5LT_FILE_IMAGE_DONT_COPY | H5LT_FILE_IMAGE_DONT_RELEASE);
        H5Fclose(fileHandle);
        std::cout<<"\r";std::cout<< i << ":" <<  numOfTests;std::cout.flush();
    }
    return 0;
}

      Thank you.

Best,
Samer Afach


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Loading...