Integrating C++ singletons into postgresql extensions???

[Stephen Woodbridge]

Hi,

I've been writing C++ code that needs to get wrapped into a postgresql
extension and it has a few singleton classes like:

1. Config object that holds some general configuration information.
2. Stats object for global stats collection.
3. Logger object for enabling/disabling debug logging to a file.
4. curlpp class for making outbound calls to an OSRM web service from
the c++ code

I can probably do without 1-3 if I had to when the code is integrated,
but they are handy to keep around in the code for when we are developing
and working outside the postgresql database. I could #ifdef then all,
but I would like to avoid cluttering the code with tons of #ifdef.

Item 4 is a little more problematic and needs to be kept around or
something else implemented to take its place. I have looked at the curl
extension and actual built another module in C that uses those ideas.,
but the curlpp is a nice OO class that encapsules and hides all the curl
stuff from the C++ code.

So my question(s) are:

1. Any insights into singletons working/not working within the
postgresql server extensions? My understanding is that these are
implemented as global static classes with a lifetime of the process and
they have no destructor, so I'm a little worried about memory leaks or
multiple queries sharing the singleton.

2. Is there another way implement the equivalent of a singleton for C++
code that has a lifetime of the query in the server?

3. What do for logging classes when integrating C++ code into postgresql?

Any help or ideas would be appreciated.

Thanks,
-Steve

[Dmitriy Igrishin]

Hello,

Post by Stephen Woodbridge
Hi,
I've been writing C++ code that needs to get wrapped into a postgresql
1. Config object that holds some general configuration information.
2. Stats object for global stats collection.
3. Logger object for enabling/disabling debug logging to a file.
4. curlpp class for making outbound calls to an OSRM web service from the
c++ code
I can probably do without 1-3 if I had to when the code is integrated, but
they are handy to keep around in the code for when we are developing and
working outside the postgresql database. I could #ifdef then all, but I
would like to avoid cluttering the code with tons of #ifdef.
Item 4 is a little more problematic and needs to be kept around or
something else implemented to take its place. I have looked at the curl
extension and actual built another module in C that uses those ideas., but
the curlpp is a nice OO class that encapsules and hides all the curl stuff
from the C++ code.
1. Any insights into singletons working/not working within the postgresql
server extensions? My understanding is that these are implemented as global
static classes with a lifetime of the process and they have no destructor,
so I'm a little worried about memory leaks or multiple queries sharing the
singleton.
2. Is there another way implement the equivalent of a singleton for C++
code that has a lifetime of the query in the server?
3. What do for logging classes when integrating C++ code into postgresql?
Any help or ideas would be appreciated.

I would have written extension in pure C. And I would get rid of
singletones. (Singletone - artificial constraints that the programmer
creates for himself. Btw, C++ - too :-) )
I would use Extension Configuration Tables to store the
configuration
http://www.postgresql.org/docs/9.4/static/extend-extensions.html#AEN57238

[Andres Freund]

Post by Stephen Woodbridge
Hi,
I've been writing C++ code that needs to get wrapped into a postgresql
1. Config object that holds some general configuration information.
2. Stats object for global stats collection.
3. Logger object for enabling/disabling debug logging to a file.
4. curlpp class for making outbound calls to an OSRM web service from the
c++ code
I can probably do without 1-3 if I had to when the code is integrated, but
they are handy to keep around in the code for when we are developing and
working outside the postgresql database. I could #ifdef then all, but I
would like to avoid cluttering the code with tons of #ifdef.
Item 4 is a little more problematic and needs to be kept around or something
else implemented to take its place. I have looked at the curl extension and
actual built another module in C that uses those ideas., but the curlpp is a
nice OO class that encapsules and hides all the curl stuff from the C++
code.
1. Any insights into singletons working/not working within the postgresql
server extensions? My understanding is that these are implemented as global
static classes with a lifetime of the process and they have no destructor,
so I'm a little worried about memory leaks or multiple queries sharing the
singleton.

There's no general problem that I can see. Postgres uses processes, not
threads, for individual connections. So any process level leak doesn't
persist for longer than the connection. And if you create the singleton
inside the individual backend, instead of in the postmaster (the parent
server) they won't be shared either.

Post by Stephen Woodbridge
2. Is there another way implement the equivalent of a singleton for C++ code
that has a lifetime of the query in the server?

Hm. Now it needs to be query lifetime, not connection lifetime? That's
slightly harder. If that's what you need probably the best way to go
about it would be to register a callback with
RegisterResourceReleaseCallback().

Post by Stephen Woodbridge
3. What do for logging classes when integrating C++ code into postgresql?

I don't really know what you want to do here. My guess is that it'd be
most appropriate to map the logging onto postgres' internal
logging. That should easily be possible.


The most important thing to be aware of here is that you need to take
*great* care that exception, *NEVER* permeate through postgres
code. Doing so will cause bad things.

Greetings,

Andres Freund

[Stephen Woodbridge]

Post by Andres Freund

Post by Stephen Woodbridge
Hi,
I've been writing C++ code that needs to get wrapped into a postgresql
1. Config object that holds some general configuration information.
2. Stats object for global stats collection.
3. Logger object for enabling/disabling debug logging to a file.
4. curlpp class for making outbound calls to an OSRM web service from the
c++ code
I can probably do without 1-3 if I had to when the code is integrated, but
they are handy to keep around in the code for when we are developing and
working outside the postgresql database. I could #ifdef then all, but I
would like to avoid cluttering the code with tons of #ifdef.
Item 4 is a little more problematic and needs to be kept around or something
else implemented to take its place. I have looked at the curl extension and
actual built another module in C that uses those ideas., but the curlpp is a
nice OO class that encapsules and hides all the curl stuff from the C++
code.
1. Any insights into singletons working/not working within the postgresql
server extensions? My understanding is that these are implemented as global
static classes with a lifetime of the process and they have no destructor,
so I'm a little worried about memory leaks or multiple queries sharing the
singleton.

There's no general problem that I can see. Postgres uses processes, not
threads, for individual connections. So any process level leak doesn't
persist for longer than the connection. And if you create the singleton
inside the individual backend, instead of in the postmaster (the parent
server) they won't be shared either.

Post by Stephen Woodbridge
2. Is there another way implement the equivalent of a singleton for C++ code
that has a lifetime of the query in the server?

Hm. Now it needs to be query lifetime, not connection lifetime? That's
slightly harder. If that's what you need probably the best way to go
about it would be to register a callback with
RegisterResourceReleaseCallback().

First, thank you for the very helpful answers. I'll think about this
some more, I might be able to get away with connection level handling,
but it is good to know what my options are.

Post by Andres Freund

Post by Stephen Woodbridge
3. What do for logging classes when integrating C++ code into postgresql?

I don't really know what you want to do here. My guess is that it'd be
most appropriate to map the logging onto postgres' internal
logging. That should easily be possible.

Ahh, sorry this is just debug info that we write to stdout when we are
testing and debug the code from the command line. There are some
packages like log4cpp and log4cxx that provide objects that can be
easily configure to change the output stream from stdout to a log file
or turn it off completely. The benefit of a logging class like this is
that you don't have to #ifdef all these statements in the code and you
can move your code from one environment to another with nothing more
than changing a parameter or two.

Post by Andres Freund
The most important thing to be aware of here is that you need to take
*great* care that exception, *NEVER* permeate through postgres
code. Doing so will cause bad things.

Yes, our C code does all the SPI and interface with postgresql and then
calls a C++ wrapper that has a try {} catch() block to guard against
this and that has been working well.

Thanks!

Best regards,
-Steve

Post by Andres Freund
Greetings,
Andres Freund

[Tom Lane]

Post by Stephen Woodbridge

Post by Andres Freund

Post by Stephen Woodbridge
3. What do for logging classes when integrating C++ code into postgresql?

I don't really know what you want to do here. My guess is that it'd be
most appropriate to map the logging onto postgres' internal
logging. That should easily be possible.

Ahh, sorry this is just debug info that we write to stdout when we are
testing and debug the code from the command line. There are some
packages like log4cpp and log4cxx that provide objects that can be
easily configure to change the output stream from stdout to a log file
or turn it off completely.

FWIW, if you are using the logging collector (highly recommended), output
to a backend process's stdout or stderr will be caught and included in the
log, though it won't have a log_line_prefix. This might be a usable
substitute for adapting your code.

regards, tom lane