Your Ad Here
It is currently Fri Nov 20, 2009 5:28 pm



Post new topic Reply to topic  [ 689 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7, 8 ... 46  Next
Author Message
 Post subject: Re: SimulationCraft
PostPosted: Sat Aug 23, 2008 11:30 am 
Shadow Priest
Joined: Wed Feb 07, 2007 6:05 am
Posts: 173
Offline

Just a simple question, is there a posibilty to specify group composition in SimCraft? I think it would be nice to simulate for example the amount of Mana gained by Improved WE compared to VT, based on how many mana user are present and stuff like that.


Top
 Profile  
 
 Post subject: Re: SimulationCraft
PostPosted: Sat Aug 23, 2008 2:44 pm 
Shadow Priest
Joined: Thu Aug 17, 2006 10:19 am
Posts: 1221
Offline

Bawi wrote:
Just a simple question, is there a posibilty to specify group composition in SimCraft? I think it would be nice to simulate for example the amount of Mana gained by Improved WE compared to VT, based on how many mana user are present and stuff like that.


Yes, you can make arbitrary party/raid configurations of any class it supports:
Druid (Balance only)
Mage
Priest
Shaman



_________________
http://code.google.com/p/simulationcraft
Top
 Profile  
 
 Post subject: Re: SimulationCraft
PostPosted: Sat Aug 23, 2008 4:19 pm 
Shadow Priest
Joined: Sun Oct 28, 2007 5:17 am
Posts: 2029
Location: EU-Jaedenar
Offline

Dedmon, I've just studied frost mages' scaling and have several questions:
1) Wtf is with this insane scaling? Compare it with: viewtopic.php?f=54&t=15727
2) Sorry for stupid question but why Spirit gives a frost mage +dps?
3) Why frost mage has improved scorch and never casts it? I was under impression that particularly Frost_Mage is supposed to be a support biatch.


Attachments:
File comment: Results of Frost Mage and Shadow Priest scaling research
scaling_results.zip [11.94 KiB]
Downloaded 13 times
File comment: Configs for Frost Mage scaling research
fm_scaling.zip [3.56 KiB]
Downloaded 12 times

_________________
Excelsior
Top
 Profile  
 
 Post subject: Re: SimulationCraft
PostPosted: Sat Aug 23, 2008 5:13 pm 
Shadow Priest
Joined: Thu Aug 17, 2006 10:19 am
Posts: 1221
Offline

Battlemaid wrote:
Dedmon, I've just studied frost mages' scaling and have several questions:
1) Wtf is with this insane scaling? Compare it with: http://shadowpriest.com/viewtopic.php?f=54&t=15727
2) Sorry for stupid question but why Spirit gives a frost mage +dps?
3) Why frost mage has improved scorch and never casts it? I was under impression that particularly Frost_Mage is supposed to be a support biatch.


(1) I haven't had time to look that closely..... but you need to be very very careful when calculating scaling values. It is important that you vary the stat of interest significantly enough to out-weigh any variance in the simulation. So.... Best results occur when the stat change is large and the iteration count is large.

(2) The holy priest in the raid buffs everybody with IDS. (And most folks are getting BoK, too.)

(3) Probably a good idea..... I have the Fireball Mage and Frostfire Mage on scorch duty, but your setup makes more sense.



_________________
http://code.google.com/p/simulationcraft
Top
 Profile  
 
 Post subject: Re: SimulationCraft
PostPosted: Sat Aug 23, 2008 5:15 pm 
Shadow Priest
Joined: Thu Aug 17, 2006 10:19 am
Posts: 1221
Offline

Just took a peek at your scaling results..... I think you are getting bit by variance.

If you are simulating just one player at a time, then you could easily increase the iterations by a factor of 100.



_________________
http://code.google.com/p/simulationcraft
Top
 Profile  
 
 Post subject: Re: SimulationCraft
PostPosted: Sat Aug 23, 2008 5:20 pm 
Shadow Priest
Joined: Sun Oct 28, 2007 5:17 am
Posts: 2029
Location: EU-Jaedenar
Offline

dedmonwakeen wrote:
(1) I haven't had time to look that closely..... but you need to be very very careful when calculating scaling values. It is important that you vary the stat of interest significantly enough to out-weigh any variance in the simulation. So.... Best results occur when the stat change is large and the iteration count is large.

I used delta of 200, but when I tested variance of FM's dps at 1000 iterations, I saw +-40. So I really fucked up.

dedmonwakeen wrote:
(2) The holy priest in the raid buffs everybody with IDS. (And most folks are getting BoK, too.)

Check raid_sole_frostmage_baseline.txt and raid_sole_frostmage_200_spi.txt. They model a single Frost Mage dpsing. The latter has 200 more spirit than prior, and simultaneously it has shitloads more dps (shitloads means something not even close to double variance).



_________________
Excelsior
Top
 Profile  
 
 Post subject: Re: SimulationCraft
PostPosted: Sat Aug 23, 2008 5:29 pm 
Shadow Priest
Joined: Sun Oct 28, 2007 5:17 am
Posts: 2029
Location: EU-Jaedenar
Offline

Dedmon, only look at this. How can we fight variance?! :shock:

iterations=100000

#1 Player=Mage_Frost DPS=2544.8 DPM=13.2 MPS=192.8/0.0
#2 Player=Mage_Frost DPS=2520.4 DPM=13.2 MPS=191.5/0.0
#3 Player=Mage_Frost DPS=2522.1 DPM=13.2 MPS=191.6/0.0
#4 Player=Mage_Frost DPS=2571.3 DPM=13.3 MPS=193.7/0.0



_________________
Excelsior
Top
 Profile  
 
 Post subject: Re: SimulationCraft
PostPosted: Sat Aug 23, 2008 5:44 pm 
Shadow Priest
Joined: Sun Oct 28, 2007 5:17 am
Posts: 2029
Location: EU-Jaedenar
Offline

Upd. This is totally unimaginable. At 10^6 iterations (each of those takes 6 minutes on my PC, so increasing by 1 more order is not an option):

#1 Player=Mage_Frost DPS=2538.0 DPM=13.2 MPS=192.5/0.0
#2 Player=Mage_Frost DPS=2560.9 DPM=13.2 MPS=193.4/0.0

Code:
Player=Mage_Frost DPS=2538.0 DPM=13.2 MPS=192.5/0.0
  Core Stats:  strength=44  agility=50  stamina=726  intellect=820  spirit=831  health=9800  mana=14443
  Spell Stats:  power=2000  hit=7.6%  crit=23.6%  penetration=0  haste=10.9%  mp5=91
  Attack Stats:  power=34  hit=0.0%  crit=4.0%  expertise=0.0  penetration=0  haste=10.9%
  Actions:
    fire_ball             Count= 15.4|18.9sec  DPS=2558.3  DPE=3706| 8%  DPET=2558  Miss=6.4%  Hit=3348  CritHit=5022|5022|34.1%
    frost_bolt            Count=109.3| 2.7sec  DPS=2242.8  DPE=5107|76%  DPET=2243  Miss=6.4%  Hit=3750  CritHit=7501|7501|42.6%
    ice_lance             Count=  8.7|33.4sec  DPS=2049.7  DPE=2960| 3%  DPET=2050  Miss=6.4%  Hit=3161  CritHit=3161|3161|81.0%
    ignite                Count=  0.0| 1.$sec  DPS= 568.1  DPE=   1| 2%  DPET=   1  Miss=-1.$%  Tick=1136
    water_elemental-frost_bolt  Count= 74.9| 3.9sec  DPS= 454.7  DPE=1137|12%  DPET= 455  Miss=17.0%  Hit=1308  CritHit=1962|1962|7.8%

RDPS=2538.0

Gains:

Procs:

    Mage_Frost:
        deferred_ignite=0|26169.3sec
        judgement_of_wisdom=37|7.7sec

Up-Times:

    Global:
        98.6% : winters_chill

    Mage_Frost:
        16.9% : fingers_of_frost
        62.1% : water_elemental

Waiting:
    All players active 100% of the time.

Performance:
  TotalEvents   = 698348526
  MaxEventQueue = 9
  SimSeconds    = 290662529
  CpuSeconds    = 409
  SpeedUp       = 710666


Code:
Player=Mage_Frost DPS=2560.9 DPM=13.2 MPS=193.4/0.0
  Core Stats:  strength=44  agility=50  stamina=726  intellect=820  spirit=831  health=9800  mana=14443
  Spell Stats:  power=2000  hit=7.6%  crit=23.6%  penetration=0  haste=10.9%  mp5=91
  Attack Stats:  power=34  hit=0.0%  crit=4.0%  expertise=0.0  penetration=0  haste=10.9%
  Actions:
    fire_ball             Count= 14.5|18.9sec  DPS=2563.5  DPE=3705| 8%  DPET=2564  Miss=6.4%  Hit=3348  CritHit=5022|5022|34.1%
    frost_bolt            Count=103.0| 2.7sec  DPS=2247.8  DPE=5107|75%  DPET=2248  Miss=6.4%  Hit=3750  CritHit=7501|7501|42.6%
    ice_lance             Count=  8.2|33.4sec  DPS=2054.7  DPE=2960| 3%  DPET=2055  Miss=6.4%  Hit=3161  CritHit=3161|3161|81.0%
    ignite                Count=  0.0| 1.$sec  DPS= 568.1  DPE=   1| 2%  DPET=   1  Miss=-1.$%  Tick=1136
    water_elemental-frost_bolt  Count= 74.9| 3.6sec  DPS= 454.7  DPE=1137|12%  DPET= 455  Miss=17.0%  Hit=1308  CritHit=1962|1962|7.8%

RDPS=2560.9

Gains:

Procs:

    Mage_Frost:
        deferred_ignite=0|25093.9sec
        judgement_of_wisdom=35|7.7sec

Up-Times:

    Global:
        98.5% : winters_chill

    Mage_Frost:
        16.9% : fingers_of_frost
        66.0% : water_elemental

Waiting:
    All players active 100% of the time.

Performance:
  TotalEvents   = 663007781
  MaxEventQueue = 9
  SimSeconds    = 273398141
  CpuSeconds    = 394
  SpeedUp       = 693904



_________________
Excelsior
Top
 Profile  
 
 Post subject: Re: SimulationCraft
PostPosted: Sat Aug 23, 2008 5:46 pm 
Shadow Priest
Joined: Thu Aug 17, 2006 10:19 am
Posts: 1221
Offline

Battlemaid wrote:
Check raid_sole_frostmage_baseline.txt and raid_sole_frostmage_200_spi.txt. They model a single Frost Mage dpsing. The latter has 200 more spirit than prior, and simultaneously it has shitloads more dps (shitloads means something not even close to double variance).


Hmmm..... Check out the "Waiting" report...... Perhaps he is running out of mana given the lack of VT, Mana Spring, Elemental Oath?

If the player does not show up in the Waiting report, that means he was chain-casting.



_________________
http://code.google.com/p/simulationcraft
Top
 Profile  
 
 Post subject: Re: SimulationCraft
PostPosted: Sat Aug 23, 2008 5:47 pm 
Shadow Priest
Joined: Sun Oct 28, 2007 5:17 am
Posts: 2029
Location: EU-Jaedenar
Offline

dedmonwakeen wrote:
Battlemaid wrote:
Check raid_sole_frostmage_baseline.txt and raid_sole_frostmage_200_spi.txt. They model a single Frost Mage dpsing. The latter has 200 more spirit than prior, and simultaneously it has shitloads more dps (shitloads means something not even close to double variance).


Hmmm..... Check out the "Waiting" report...... Perhaps he is running out of mana given the lack of VT, Mana Spring, Elemental Oath?

If the player does not show up in the Waiting report, that means he was chain-casting.

infinite_mana=1

Upd. Check the details above



_________________
Excelsior
Top
 Profile  
 
 Post subject: Re: SimulationCraft
PostPosted: Sat Aug 23, 2008 5:55 pm 
Shadow Priest
Joined: Sun Oct 28, 2007 5:17 am
Posts: 2029
Location: EU-Jaedenar
Offline

I didn't have a chance yet to dig into used RNG library, but maybe you can clarify what distribution of a variate it uses? Using some facts from theory of probability we could possibly find out distribution of DPS for some spec and check how it changes with increasing of iterations.



_________________
Excelsior
Top
 Profile  
 
 Post subject: Re: SimulationCraft
PostPosted: Sat Aug 23, 2008 6:17 pm 
Shadow Priest
Joined: Thu Aug 17, 2006 10:19 am
Posts: 1221
Offline

Battlemaid wrote:
Upd. This is totally unimaginable. At 10^6 iterations (each of those takes 6 minutes on my PC, so increasing by 1 more order is not an option):
#1 Player=Mage_Frost DPS=2538.0 DPM=13.2 MPS=192.5/0.0
#2 Player=Mage_Frost DPS=2560.9 DPM=13.2 MPS=193.4/0.0


( 2561 - 2538 ) / 2538 = 0.009 => 1% variance

To be honest, that's not bad. It used to take me 48hrs of processing time to generate the scaling tables on wiki.shadowpriest.com......

What I used to do was spend a great deal of effort getting a solid baseline (super-high iterations).
This would essentially cut my variance error in half when comparing DeltaGear to BaselineGear.

Unfortunately, I've plucked all the low-hanging performance fruit. There is a significant cost associated with an architecture designed to support every class.....

All the proc-heavy classes (I consider crit to be a "proc") like Destro-Warlocks are harder to model because the variance is exacerbated.



_________________
http://code.google.com/p/simulationcraft
Top
 Profile  
 
 Post subject: Re: SimulationCraft
PostPosted: Sun Aug 24, 2008 11:02 am 
Shadow Priest
Joined: Thu Aug 17, 2006 10:19 am
Posts: 1221
Offline

I had an epiphany while creating the subject matter Centrelink is so fond of tricking folks into looking at........

Halfway through the first simulation iteration I extrapolate the Boss max health based upon damage so far.
I use this health for the remainder of the first sim as well as every subsequent iteration.

This is necessary to properly model talent/skills like Molten Fury and Execute.

One unnecessary form of variance is the fact that folks get their last cast canceled midway through when the boss dies. Consider the Mage chain-casting a 3sec Fireball. Depending upon when the cast is interrupted this represents a 0% to 1% variance.

I'm going to change the code such that everybody finishes their last cast always..... which means that each player will have a slightly different fight time.

Spell crit is still the largest source of variance..... but at least this is a start.



_________________
http://code.google.com/p/simulationcraft
Top
 Profile  
 
 Post subject: Re: SimulationCraft
PostPosted: Sun Aug 24, 2008 2:15 pm 
Shadow Priest
Joined: Sun Oct 28, 2007 5:17 am
Posts: 2029
Location: EU-Jaedenar
Offline

Quote:
Spell crit is still the largest source of variance

For me at 10^5 of iterations procs (including crit) stabilize with +/-0.05% precision (max difference for me was 0.1%). At 10^6 of iterations in a handful tests I didnt witness proc chances to vary even with 1 digit.

I believe that we can eliminate a significant part of variance:
* If you fix the "last cast" problem
* If sim is run with average_dmg=1

Under these conditions if we can guarantee that crit (lets limit ourselves to the only proc atm) variates within +/-0.1%, then we can guarantee that dps variates within +/-0.1%, which is 3-4 dps for WotLK. This still brings ~10 dps [delta-base] variance which will severely affect scaling analysis.

Here are the precisions we can guarantee for scaling numbers depending on scaling analysis step (x-axis) and guaranteed dps variance (y-axis). As one can see, under current conditions we CAN NOT perform quick analysis that could guarantee us even 1 significant digit of the result while using more or less precise step of research. We could guarantee 1 digit only with variance kept within 0.01%. Two digits can be guaranteed if variance is at max 0.001%.

Image

To give even more information for thought, I've calculated dps of a single frost mage for 5 times for each of the cases: [10^3, 10^4 and 10^5 iterations] X [average_dmg=0, average_dmg=1]. No spreadsheet for now, don't have the time.


Attachments:
Variance project.zip [30.29 KiB]
Downloaded 12 times

_________________
Excelsior
Top
 Profile  
 
 Post subject: Re: SimulationCraft
PostPosted: Wed Aug 27, 2008 1:15 pm 
Shadow Priest
Joined: Sun Oct 28, 2007 5:17 am
Posts: 2029
Location: EU-Jaedenar
Offline

Actually all pesky variance comes from procs (given average_dmg=1).

Before one may start to saying something about crit. Latter is also a PROC, a Programable Random OCcurrence. Same goes for resistances and dodges that can be modelled as a proc that adversely affects incoming damage.

How do we model a proc?

From a mathematical standpoint it's described by:
1) Trigger that might activate the proc (some gameplay event)
2) Occurrence pattern (includes chance to proc and internal cooldowns)
3) Things that happen when the proc activates (and possibly deactivates)

At this very moment I can't find a way to address variance in its general case.

Of course, one can replace proc model with some average values when a single thingie is modelled. However, modelling synergy of multiple procs is where analytic models start to suck. This is why SimCraft is necessary. This is why we have to tolerate variance. This is why we have to wait 48h to get only that damned 0.015 precision.

However, sometimes we can replace random thingies that happen with their statistically averaged equivalents, while not losing comprehensiveness of the simulation. As I vision it, we can legally do so when and only when a given random event doesn't have any side effects.

Example number only, since I have to go home from work :(

Crit.

Critical strike is a rather unique proc, because it does two things. First of those (Part 1) is dealing additional damage, second (Part 2) is triggering all those sweet on-crit talents and procs from gear. Devariating part two looks like a very difficult task to me since all those procs might cause other procs which can cause other procs. However additional damage dealt doesn't spawn side-effects and can be dealt with.

To 100% devariate part one we need to know two things: A) analytically calculated mathematical expectation of the event's occurrence model and B) event's value. Thanks to average_dmg=1, part B is deterministic.

Part A is too, since crit% chance doesn't change during the raid and can be calculated apriori before the fight. So instead of having each time different crit chances and different resulting dps for so many different spells, we'll bake analytically prognozed crit damage into our results, effectively decomposing crit into 2 procs: Part 1 and Part 2, one of which becomes totally deterministic.

I wish the previous paragraph was true, tho it illustrated my main idea.

To my dismay, there exist procs (e.g. Destruction or Elemental Oath) that dynamically change crit% chance during the raid. I think we can model them independently from the main raid, and calculate mathematical expectation of the crit-proc pattern. Of course, it won't be 100% precise, but due to task decomposition we can find it much faster = with much better precision that SimCraft can guarantee for raid dps. Using pre-calculated crit chance with high precision, we can bake it into our results, guaranteeing lower (but still better than original) precision for the whole picture.



_________________
Excelsior
Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 689 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7, 8 ... 46  Next

All times are UTC - 6 hours [ DST ]


Who is online

Users browsing this forum: Google Adsense [Bot], Spyker and 10 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Latest from LightNatured

Latest from WowInsider

Powered by

Protected by Anti-Spam ACP
Copyright © 2003 - 2009 by Bryghtpath LLC - All Rights Reserved
Design by Kiss My Code with Icons from World of Warcraft Styles.