Exploding die
An exploding die is a term used when achieving certain results on a roll allows additional rolling to achieve more significant effects. For example, in new World of Darkness, rolling a 10 adds a success to the skill attempt in question and grants an extra roll of the d10 to the player, allowing more successes than usually possible to be achieved. In L5R, any tens rolled are rerolled and the new number is added to the previous 10 to determine the result of the dice throw (if this is also a ten, the process repeats, allowing lucky players to achieve truly ridiculous results.)
In the strategy/RPG game Mekton, rolling a 10 on a skill check causes the die to explode: roll again, and add ten. This can repeat as many times as the player rolls 10. Rolling a 1 causes the die to collapse: the player rolls the die again and subtracts the number from their skill total. Also, the Location Damage chart to determine where a shot hits will upgrade it to a critical. Rolling another 10 will force the opposing player to save (and take 50% damage to the affected area) or die instantly if the shot breaches armour, while a 8 or 9 will cause a "Cinematic" critical hit, which causes a fixed amount of unavoidable damage and often has other undesirable effects (such as missing a turn or being instantly removed from play).
In Dark Heresy, rolling a 10 when calculating damage triggers "Righteous Fury"; the attacker rerolls their to-hit percentile, and if it is a success again they may roll that damage die again. If that ALSO comes up a 10, the Emperor must fucking love you, as you get to reroll the damage die yet AGAIN (you don't have to reroll to hit anymore) until you no longer get a 10. This can be especially beneficial for weapons that roll more than one die (such as Heavy Bolters, which are 2d10) or weapons that have the Tearing Quality (where you roll damage twice and keep the better result), since you have a greater chance to trigger Righteous Fury. Combine both and you have some pretty good chances, as high-BS Heavy Bolter Devastators in Deathwatch routinely prove. The epic saga of Grendel is based on such epic exploding rolls.
In the Strategy miniatures game Mechaton Titan, rolling a double six on any weapon causes rendering, adding an additional d4 damage: this signifies secondary explosions, such as fuel cells or ammunition exploding inside the structure. Apocalypse similarly has an exploding D6 for damage rolls on superheavy vehicles. And of course, 40k has Rending, albeit it allows to roll only a D3 for every six on an armor penetration roll (used to be another D6 in 4E, which was devastating).
In Deadlands, rolling an "Ace" (the highest value the die allows) on any die allows re-rolling it and adding the new value.
It has nothing to do with joke dice that actually explode.
Math stuff
How do you get the average of an exploding die? If you ask a friend who's good at math, he'll tell you a long explanation about Sequences and Limits, of which you'll forget half. What you'll remember is the conclusion:
The average of an exploding Dn tends to the average of a regular Dn (which is (n+1)/2) multiplied by n/(n-1)
This means that for an exploding D6, the average is 3.5 * 6/5 = 4.2, while an exploding D10 bumps its average from 5.5 to 6.11