I've regularly had this issue also. Both with my 2003 Grand Am and my 2005 G6 GT. With the Grand Am I just figured it was due to the aged battery not providing quiet enough juice to start up right away but with the G6 seems like it might have to due with the fuel pump priming up. The G6 does it most frequently when I drive the car somewhere, shut it off, then start it back up shortly after (for example, stopping at the post office -- the car will be off for 5 minutes or so and will somewhat struggle to get going again). I call it the Warm Engine Syndrome.
Although it's a bit embarrassing for me, the G6 has never failed to start, it just takes a lot of cranking to get going sometimes. My hypothesis is that the fuel pump takes time to get fuel back up to the engine.
One time, on a cool night of about 40 degrees I used the remote start and the car took about 15 seconds to start. It just keep cranking and cranking and when it did start a puff of smoke came out of the tailpipe. That is the worst I've seen but I was impressed that the car didn't quit and did manage to start itself after all that. This again was with my Warm Engine Syndrome.
I don't know if the car ever has trouble with a cold engine, I usually remote start the car from a distance, like when I'm inside my apartment, and it'll always been running when I get outside.