The short answer is, yes you can.
Modern anti-materiel rifles are basically sniper rifles with a little more power. They typically fire .50 caliber or 20mm armor-piercing rounds. Machine guns and autocannons of these calibers are commonly used as vehicle-mounted weapons. It isn't too hard to imagine these weapons being modified to be hand-held by someone or something 3 times larger than a normal person.
As you suggest in your question, these weapons are not effective against tank armor, but they can damage less heavily armored vehicles. The standard ammuntion would probably be of the kinetic energy penetrator type. These have a high-density core of a tungsten alloy or depleted uranium with a lightweight outer shell that is discarded when the round is fired. This means that the projectile moves faster than a regular bullet and transfers the impact to a smaller area, allowing them to pierce through armor.
As for grenades, I'm with DaaaahWhoosh—there's not much reason for armored mechs to use them. At least, assuming that regular infantry use powered armor, and there aren't many unarmored targets to use them against. There could be use for rocket-propelled grenades, though, as they can be used against armor. There are HEAT rounds, which use a shaped charge to propel a stream of metal at hypersonic speeds through armor. Also used are HESH rounds, which use a plastic explosive that spreads across an armored surface before detonating to send a shockwave through the armor and damage things inside without piercing the armor. If there is a use for shrapnel in your setting, HEAT rounds can be modified to become high-explosive dual purpose weapons by adding shrapnel elements to the outside of the explosive charge.
Edit: If you really want explosive handheld grenades, your mechs are probably big enough (and your technology advanced enough) to consider low-yield nuclear weapons. The smallest device known to have been made, the W54, weighed about 50lbs and had a yield of about 10 tons of TNT. According to the Wikipedia page linked above, it could destroy an area of two city blocks, which works out to a radius of about 250 feet. So, it is very destructive but it still seems plausible that a mech could throw it far enough to be safe. Later versions had up to a 250 ton yield, which might exceed the "throwable" range, though.