It all depends on the health insurance policy that you chose to continue under the COBRA insurance law.  Each group health insurance company has different policies so you will want to call your insurance provider.  Under most policies you will be allowed to add your baby once you give birth.