Zone – 4B, Sun for most of the day, Tree is in a sheltered near to house.
I noticed that there are many holes on the trunk of the tree. Just below where the tree starts branching out the bark is falling off. I have put on horticultural glue but I’m not sure if it is making it worse.
While it is normal for some trees to lose bark as they mature and grow, this specific area looks like it could be due to damage of some sort. Damage can be caused by animals, pests, weather patterns like extreme heat and drought, mechanical accidents from tools etc. Have you noticed any bird activity in the tree? The sizable holes surrounding the wound could be caused from birds, perhaps woodpeckers seeking insects. Ginkgo trees are known for being largely insect tolerant due to natural acids they produce that act like a repellant. According to the University Redlands website there are insects that have managed to link themselves to ginkgos, however most have been detected associated with the foliage.
It can be very difficult to make a definitive diagnosis from a photo. Based on the one you have submitted of your mature ginkgo, I would strongly recommend having a professional arborist have a look at your tree in person. To find a certified arborist in your area you can do a search for one through the Landscape Ontario website here https://landscapeontario.com/search
Best of luck with your tree.