Hey, Ashlee - welcome to Splitcoast! Before I moved across the country, I had a lot of my stamping supplies in a utility closet. Not everything, but the things I didn't want to be looking at all the time... Here's a shot:
I obviously had my laptop in there, but you could switch that out easily enough. The photo cases on the upper shelf had all of my embellishments in them, sorted by color. The little red and white boxes are actually drawers from units similar to the one on the lower left. They had embossing powders and glitter in them. The ink pads are in the units that the drawers came out of, as well as in CD crates. Other small tools and things were in the purple drawers.
On the shelves above where the photo cuts off were specialty papers, stencils, dies and such-like, and I had an Iris cart under the "desk" on the left with ribbons, flowers, and some stamps (peg alphabets, self-inking, etc.), and more tools. All of my 6" x 6" and "mat stack" papers were in a small crate on top.
I don't know how large your closet is, or if you're trying to put ALL of your supplies in it, but it's do-able to a large degree. That's especially true if your stamps are photopolymer and/or unmounted rubber.
Some things are just bulky - like punches - and you kind of have to brainstorm an out-of-the-box storage solution for them if you want your entire collection to be contained in the closet. Over-the-door wire racks or shoe pouches have worked well for others. I'm a fan of media towers, myself, but you'd have to determine if you had room...
I'm sure that others will have great ideas for you, and I'm not sure how helpful I've been, but any/all suggestions help the wheels turn, right? Good luck with that!