Thinking about "what is a programming language" gets more interesting whenever you also ask "what is an interpreter" and you eventually realize interesting things like how a user interface is an interpreter for the language of human actions.

Building languages used to be a bit more common, I think, since there are some tools specifically for this purpose (the program "yacc" even stands for "yet another compiler compiler"). I know that I have built some tools using those programs to come up with usually more constrained versions of existing languages to make programs easier to prove correct. I always figured that I would make my own language on top of C using OctoberProject as the driving force if it ever got big enough in user base that we didn't want to rely on the JVM being available (since it is owned by a company). I like how C works but there are some additional bits of static verification I would like to see and some additional idioms I would like to add.