I don’t know when the first comic book horror story appeared. When evaluating old stories one may have different definitions of what “horror” is. I tend to think of comic book horror as it was found in EC Comics and their myriad imitators. In creating their horror comics Bill Gaines and Al Feldstein of EC supposedly talked about a radio program they both liked, The Witch’s Tale, and that was their springboard. The Witch’s Tale featured Old Nancy as the witch. The old witch in this story does not have a name, just a story to tell. I presume one could imagine any witchy-sounding cackling noises.
“Weird Tales, The Tale of the Haunted Inn” is from Hit Comics #1 (1940). It’s likely they swiped the witch from the radio show, and the title of the feature from Weird Tales, the pulp magazine.