I want to start by saying that I hope your boyfriend appreciates that you're aware of the situation and you want to try to change things. When it comes to who initiates sex it's probably not true that it's the thought that counts, but I'd say the thought counts for something.
Sometimes when we're having problems with sex in a relationship it's because sex is a problem. Sometimes it's other stuff from our relationship that is sort of filtering into our sex lives, and the awkwardness or anger or stuck feeling is actually about something other than sex, but it plays out in the bedroom. Sometimes it's a bit of both.
You may find some inspiration in my guide to initiating sex. But it's not a check list and I don't think there's a list of ten tips that you can follow in order to start initiating sex in a way that will get you the desired outcome and feel good. Contrary to how it looks in mainstream film and television or pornography, starting something up from a place of genuine sexual desire takes some nerve and some practice. You're really putting yourself out there, and even in a long term relationship, it can feel like if it doesn't go well, it's going to go terrible.
I say this just to encourage you to give yourself some credit and be patient with yourself. Dealing with our sexual shame isn't fast work, but it's worthwhile. If you want to have sex with your boyfriend then you're most of the way there. The rest is about finding a way that works for you. You mention that you tried lingerie but he likes a t-shirt. I want to ask; what do you like? What clothes make you feel sexual and assertive? What ways of communicating do you have that allow you to take up space and feel in control? Even if you don't usually do this with sex, you probably do it in other parts of your life, and you can bring that energy into your sex life.
Initiating sex doesn't mean you have to do all the work or be in control the whole time (what fun would that be?) But it does mean that you're taking responsibility for getting things going. Stereotypes and gender roles dictate that only some of us should do this, or only some of us do it well. But your boyfriend doesn't want to have sex with a stereotype, he wants to have sex with you!