Is Removing A Cast From A Fractured Hand Before The Healing Period Safe?

Hello! I sustained a 4th metacarpal base fracture almost 4 weeks ago and was placed in a cast. I ve been taking calcium, magnesium. Zinc, vitamin C, multi vitamins, creatine supplements and CBD tincture almost daily since the fracture. I have full mobility in all fingers and a nearly normal squeezing grip plus able to move my wrist quite a bit inside my cast, all without pain. Would it be considered even remotely safe to remove my cast early keep it wrapped with tape and and maintain limited movement and strain on the hand for the next few weeks?