Kitten Punctured Index Finger. Got Rabies Shot. Does It Have Side Effects?

Yes, Thank you. Today, while catching a feral kitten to take to a rescue, I startled it and it punctured my index finger. I called my Dr. and they told me to go to the ER for the rabies shot which I did. Later today, we received a call from the board of health telling us that if I want to stop the shots, I must surrender the kitten to be euthanized and tested. Since I already began the shots am I putting myself at risk for serious side effects if I continue the rabies protocol. I'd rather continue with the shots than see this little kitten destroyed. I'm 99.9% sure it does not have rabies, but even if this is so, I already began the shots. Is there harm to me involved continuing the shots? Thanks for your response.