When my baby was a couple months old he was happy when anyone would hold him. Now that he is older he is more cautious about who he will let hold him. This is called Stranger Anxiety, and quite a few babies experience this.
Sometimes in public I will try to put him down and he will start crying, or unfamiliar people at church will try to hold him and he will cry. Even putting him in the Nursery during church turns into a battle. But not to worry mothers of babies who cling to you like glue. This is just a phase and baby will get over it just as soon as baby became afraid of these strangers.
A lot of times baby will scream when you are trying to leave him with someone, but as soon as your gone or a couple minutes afterwards baby will quit crying and be just fine. If your worried about baby crying or leaving your baby while crying, don't be. Take a cell phone with you and periodically cheek in with your baby sitter to see if baby is okay.
Another way to help baby get over stranger anxiety is to let your company know that baby is shy right now and that they should try to warm up to baby first before trying to hold baby. While baby is sitting in your lap have your company tickle your baby or play peek-a-boo with baby, and soon your baby will warm up to your company. Don't ever force baby to go to someone, this could prolong the stranger anxiety.
It is rare but there are some babies that have severe lack of mommy syndrome, usually this occurs in breastfed infants, and it doesn't matter who baby is left with weather it is daddy, or Grandma or Grandpa. Baby only wants mommy. But not to worry baby will get over this fear as well. My suggestion is that for a month or so until baby gets over the whole I can't go on without my mom routine, you the mom stick around as much as possible. If you do leave try to go when baby is napping.
You can help baby get over the I want my mommy fear by having daddy get more involved. Daddy can pick baby up when baby wakes up from naps. In the middle of the night, dad can go get the baby and bring baby to mom to breastfeed and then daddy can put baby back in the crib. Baby will get used to daddy and it won't be a big issue when mommy wants to leave for a couple hours.