Two Years Ago I Had Several PEs Caused By Tamoxifen.

Two years ago I had several PEs caused by tamoxifen. That was September of 2014. My only symptoms were…
- out of breath going up and down the steps & when talking
- no pain or discomfort in my chest
- no pain in my legs
My doctor told me to stop taking the tamoxifen immediately.
Starting yesterday, I began to get out of breath just walking from the living room to the kitchen in my townhouse. (It is a very short distance .) I walked three blocks to the store today… because the weather was so nice. I began to huff and puff in less than half a city block of walking. I continued walking and had to stop two times in less than four city blocks on the way there and three times on the way back.
I enjoy walking. I am about 30lbs. over weight and 'addicted to sugar'. I walk slower because my right knee was replaced last November but I am doing well and started to walk more and faster. Just this week I eliminated all obvious sugar from my diet.
I take medicines for high blood pressure and cholesterol. Should I be concerned about another embolism?
Should I go to my Oncologist, who cared for me during the PE? Should I go to my primary care Dr. who is new? Thank you for any direction you can provide.