The ytram is a frog-like creature found on Book Assembly Island (the island with the boiler and Gehn's laboratory) in Riven. It was used by the Moiety to make "poison" darts to knock out enemies and prey when hunting. Gehn extracted the same poison serum to create the juice he smoked in his pipe.
According to From Myst to Riven and the German Riven Strategy Guide, the frog got its name from Marty O'Donnell, who was rather desperately trying to get involved with RIven in some way. While he (and Tim Larkin) did eventually become involved in the game's sound design, the Cyan team also gave him the honor of having his name put into the game ("ytram" is "Marty" backwards).
Interestingly the call of this frog can be heard in Kadish Tolesa, suggesting perhaps that it may have been written in to Riven by Gehn, either on purpose or through accidental copying.