Kerry Osborne's Oracle Blog

Autotrace Lies Too!

osborne — Mon, 15 Feb 2010 20:25:22 +0000

I posted earlier on Explain Plan’s tendency to spread rumors (Explain Plan Lies). Autotrace is another commonly used tool that suffers from the same basic character flaw (i.e. it lies too). The reason is simple. It uses the Explain Plan command under the covers.

Here’s a quick demonstration:

^?View Code NONE

> sqlplus "/ as sysdba"
 
SQL*Plus: Release 11.2.0.1.0 Production on Mon Feb 15 11:51:27 2010
 
Copyright (c) 1982, 2009, Oracle.  All rights reserved.
 
 
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
 
SYS@LAB112> @flush_pool
 
System altered.
 
SYS@LAB112> @avgskew_X
SYS@LAB112> var X varchar2(2);
SYS@LAB112> exec :X := 'N';
 
PL/SQL procedure successfully completed.
 
SYS@LAB112> -- should cause index access due to histogram on col4
SYS@LAB112> 
SYS@LAB112> set autotrace on
SYS@LAB112> select /* test autotrace */ avg(pk_col) from kso.skew a where col4 = :X
  2  /
 
 AVG(PK_COL)
------------
961866206.84
 
1 row selected.
 
 
Execution Plan
----------------------------------------------------------
Plan hash value: 568322376
 
---------------------------------------------------------------------------
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |     1 |    24 | 28420   (2)| 00:05:42 |
|   1 |  SORT AGGREGATE    |      |     1 |    24 |            |          |
|*  2 |   TABLE ACCESS FULL| SKEW |    10M|   244M| 28420   (2)| 00:05:42 |
---------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   2 - filter("COL4"=:X)
 
 
Statistics
----------------------------------------------------------
        737  recursive calls
          0  db block gets
        250  consistent gets
          0  physical reads
          0  redo size
        444  bytes sent via SQL*Net to client
        420  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          7  sorts (memory)
          0  sorts (disk)
          1  rows processed
 
SYS@LAB112> 
SYS@LAB112> set autotrace off
SYS@LAB112> 
SYS@LAB112> select sql_id, child_number, sql_text from v$sql where sql_text like '%test autotrace%'
  2  /
 
SQL_ID        CHILD_NUMBER
------------- ------------
SQL_TEXT
-----------------------------------------------------------------------------------------------------------------------------------------------------------
9hv3fa42xhcm1            0
EXPLAIN PLAN SET STATEMENT_ID='PLUS4294967295' FOR select /* test autotrace */ avg(pk_col) from kso.skew a where col4 = :X
 
68zbpw01f4gh5            0
select sql_id, child_number, sql_text from v$sql where sql_text like '%test autotrace%'
 
2h0ajqmn8zgww            0
select /* test autotrace */ avg(pk_col) from kso.skew a where col4 = :X
 
 
3 rows selected.
 
SYS@LAB112> 
SYS@LAB112> select * from table(dbms_xplan.display_cursor('2h0ajqmn8zgww','','typical'))
  2  /
 
PLAN_TABLE_OUTPUT
-----------------------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  2h0ajqmn8zgww, child number 0
-------------------------------------
select /* test autotrace */ avg(pk_col) from kso.skew a where col4 = :X
 
Plan hash value: 1946853647
 
------------------------------------------------------------------------------------------
| Id  | Operation                    | Name      | Rows  | Bytes | Cost (%CPU)| Time     |
------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT             |           |       |       |    25 (100)|          |
|   1 |  SORT AGGREGATE              |           |     1 |    24 |            |          |
|   2 |   TABLE ACCESS BY INDEX ROWID| SKEW      |  3170 | 76080 |    25   (0)| 00:00:01 |
|*  3 |    INDEX RANGE SCAN          | SKEW_COL4 |  3170 |       |     8   (0)| 00:00:01 |
------------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   3 - access("COL4"=:X)
 
 
20 rows selected.

So I flushed the shared pool, executed my statement, and autotrace says it did a full table scan on my Skew table. But it completed in less than a second - so I doubt that’s true. Note also that autotrace says I only did 250 lio’s (the table actually has over 150K blocks).

Next we looked in v$sql for any statements that contained the string “test autotrace”. We found 3 statements:

the current statement doing the lookup in v$sql (68zbpw01f4gh5)
the actual statement that we were looking for (2h0ajqmn8zgww)
and an Explain Plan statement (9hv3fa42xhcm1)

Finally, we looked at the plan that statement 2h0ajqmn8zgww had actually used with dbms_xplan.display_cursor. It shows that the statement actually did an Index Range Scan, not a Full Table Scan.

By the way, this example is on 11gR2, but it behaves the same way in 10g.

So there you have it. Autotrace lies too! (at least it can, on the plan part of the output, due to it’s reliance on the Explain Plan command)

Update: I just ran across this post by Tom Kyte that precedes mine by about 3 years. Better later than never I guess.

GATHER_PLAN_STATISTICS

osborne — Tue, 09 Feb 2010 17:02:43 +0000

Hey, I just thought of something. One of the things I do fairly regularly when struggling with a SQL statement that is not behaving goes like this:

create a copy of the statement in a test script
add the GATHER_PLAN_STATISTICS hint
duplicate all the bind variables (with values used at parse if possible)
run it
run xplan on it

This provides the ability to see how the optimizer’s estimates line up with reality using the DBMS_XPLAN.DISPLAY_CURSOR function with the ALLSTATS format option. It’s very easy to see where the optimizer is having problems by comparing the A-Rows column (actual rows) with the E-Rows column (estimated rows). Anywhere there is a big difference (think orders of magnitude), there is a potential problem. Well actually it’s a little more complicated than that, because the A-Rows are cumulative while the E-Rows are not. You have to multiply the E-Row by Starts (or divide A-Rows by the number of executions) in order to compare apples to apples. If you need a little more info see this post by Jonathan Lewis or this one by Greg Rahn. Here’s how the output looks in case you haven’t seen it before:

^?View Code NONE

SYS@LAB1024> @fs
Enter value for sql_text: %gather_plan%
Enter value for sql_id: 
 
SQL_ID         CHILD  PLAN_HASH      EXECS     AVG_ETIME      AVG_LIO SQL_TEXT
------------- ------ ---------- ---------- ------------- ------------ ------------------------------------------------------------
dvfumy503zfnw      0 1660344652          3           .00           25 select /*+ gather_plan_statistics */ t1.x, t2.y from kso.t1
                                                                      t1, kso.t2 t2 where t1.y = t2.x and rownum < 10
 
 
SYS@LAB1024> !cat dplan_allstats.sql
set lines 180
select * from table(dbms_xplan.display_cursor('&sql_id','&child_no','allstats  +peeked_binds'))
/
 
SYS@LAB1024> @dplan_allstats
Enter value for sql_id: dvfumy503zfnw
Enter value for child_no: 
 
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  dvfumy503zfnw, child number 0
-------------------------------------
select /*+ gather_plan_statistics */ t1.x, t2.y from kso.t1 t1, kso.t2 t2 where t1.y = t2.x and
rownum < 10
 
Plan hash value: 1660344652
 
--------------------------------------------------------------------------------------------------------
| Id  | Operation                     | Name         | Starts | E-Rows | A-Rows |   A-Time   | Buffers |
--------------------------------------------------------------------------------------------------------
|*  1 |  COUNT STOPKEY                |              |      3 |        |     27 |00:00:00.01 |      75 |
|   2 |   NESTED LOOPS                |              |      3 |      9 |     27 |00:00:00.01 |      75 |
|   3 |    TABLE ACCESS FULL          | T1           |      3 |  53629 |     27 |00:00:00.01 |      15 |
|   4 |    TABLE ACCESS BY INDEX ROWID| T2           |     27 |      1 |     27 |00:00:00.01 |      60 |
|*  5 |     INDEX UNIQUE SCAN         | SYS_C0014104 |     27 |      1 |     27 |00:00:00.01 |      33 |
--------------------------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   1 - filter(ROWNUM<10)
   5 - access("T1"."Y"="T2"."X")
 
 
23 rows selected.

Just a quick note on the correlation between STARTS, E-ROWS, and A-ROWS. I picked a bit of a bad example due to the “rownum < 10" predicate. It causes processing to stop on the full table scan after 9 rows are retrieved. But it does show a mismatch in the number of estimated rows and the number of actual rows. Keep in mind that the actual rows are cumulative, so it's the total rows this step has handled regardless of the number of executions. This statement has been executed 3 times (see the 3 under Starts on line 1). And the full table scan was executed (Started) 3 times (once per execution). So the optimizer was expecting 53629 rows for each execution, but got on average 9 rows per execution. Note also that the Starts column is the actual number of times the operation was executed, not the number of times that the optimizer thinks it will be executed. Here's another little bit of output to show that Starts is actual, not estimated:

^?View Code NONE

 
SYS@LAB1024> -- here's a query without the "rownum<10" predicate
SYS@LAB1024> @fs
Enter value for sql_text: %t1_nl_join_agg.sql%
Enter value for sql_id: 
 
SQL_ID         CHILD    PLAN_HASH        EXECS     AVG_ETIME      AVG_LIO SQL_TEXT
------------- ------ ------------ ------------ ------------- ------------ ------------------------------------------------------------
a36w6pfkgp2sy      0    174109304            1          3.77       53,745 select /*+ gather_plan_statistics */ avg(x*2) from ( select
                                                                          /* t1_nl_join_agg.sql */ t1.x, t2.y from kso.t1 t1, kso.t2
                                                                          t2 where t1.y = t2.x )
 
 
1 row selected.
 
SYS@LAB1024> @dplan_allstats
Enter value for sql_id: a36w6pfkgp2sy
Enter value for child_no: 
 
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  a36w6pfkgp2sy, child number 0
-------------------------------------
select /*+ gather_plan_statistics */ avg(x*2) from ( select /* t1_nl_join_agg.sql */ t1.x,
t2.y from kso.t1 t1, kso.t2 t2 where t1.y = t2.x )
 
Plan hash value: 174109304
 
-------------------------------------------------------------------------------------------------------
| Id  | Operation           | Name         | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  |
-------------------------------------------------------------------------------------------------------
|   1 |  SORT AGGREGATE     |              |      1 |      1 |      1 |00:00:03.77 |   53745 |    169 |
|   2 |   NESTED LOOPS      |              |      1 |  53629 |  25994 |00:00:02.94 |   53745 |    169 |
|   3 |    TABLE ACCESS FULL| T1           |      1 |  53629 |  53629 |00:00:00.80 |     114 |    105 |
|*  4 |    INDEX UNIQUE SCAN| SYS_C0014104 |  53629 |      1 |  25994 |00:00:02.41 |   53631 |     64 |
-------------------------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   4 - access("T1"."Y"="T2"."X")
 
 
21 rows selected.
 
SYS@LAB1024> -- the stats are 100% correct on the full table scan (line 3), 
SYS@LAB1024> -- but off by 2X on the NL join (25994 vs. 53629 on line 2)
SYS@LAB1024> -- so let's mess up the stats on T1 to see if Starts is optimizer guess or actual
SYS@LAB1024> -- set rows in T1 to 1234
SYS@LAB1024> 
SYS@LAB1024> exec dbms_stats.set_table_stats(ownname => 'KSO', tabname => 'T1', numrows => 1234);
 
PL/SQL procedure successfully completed.
 
SYS@LAB1024> @flush_sql
Enter value for sql_id: a36w6pfkgp2sy
 
PL/SQL procedure successfully completed.
 
SYS@LAB1024> @t1_nl_join_agg.sql
 
    AVG(X*2)
------------
108794.25383
 
1 row selected.
 
SYS@LAB1024> @dplan_allstats
Enter value for sql_id: a36w6pfkgp2sy     
Enter value for child_no: 
 
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  a36w6pfkgp2sy, child number 0
-------------------------------------
select /*+ gather_plan_statistics */ avg(x*2) from ( select /* t1_nl_join_agg.sql */
t1.x, t2.y from kso.t1 t1, kso.t2 t2 where t1.y = t2.x )
 
Plan hash value: 174109304
 
----------------------------------------------------------------------------------------------
| Id  | Operation           | Name         | Starts | E-Rows | A-Rows |   A-Time   | Buffers |
----------------------------------------------------------------------------------------------
|   1 |  SORT AGGREGATE     |              |      1 |      1 |      1 |00:00:03.63 |   53745 |
|   2 |   NESTED LOOPS      |              |      1 |   1234 |  25994 |00:00:03.20 |   53745 |
|   3 |    TABLE ACCESS FULL| T1           |      1 |   1234 |  53629 |00:00:00.86 |     114 |
|*  4 |    INDEX UNIQUE SCAN| SYS_C0014104 |  53629 |      1 |  25994 |00:00:02.41 |   53631 |
----------------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   4 - access("T1"."Y"="T2"."X")
 
 
21 rows selected.
 
SYS@LAB1024> -- note that Starts on the Index Lookup inside the NL Join is 53629 (line 4)
SYS@LAB1024> -- this is the number of times that operation was actually executed,
SYS@LAB1024> -- not 1234, which is the number of times the optimizer thought is would be executed
SYS@LAB1024> -- so Starts and A-rows are actual values, E-Rows is the optimizer estimate
SYS@LAB1024> -- (at least for NL Join)

Back to the topic at hand, my normal steps (i.e. generate test script, add GATHER_PLAN_STATISTICS hint, etc…)

I have a script that helps generate the test script (see this post: Creating Test Scripts with Bind Variables). But there are definite drawbacks to this approach. It can be difficult to duplicate the exact environment that the optimizer sees when running the production code, even if you are testing on the production instance. The bottom line is that you have to be pretty careful to make sure that your test script is really doing the same thing that the production code is doing.

Which brings me to my idea (finally)! Here’s a quick way to get extended plan statistics for any statement without modifying the code or creating a test version. Drum roll please ….

Just create a one line SQL Profile (using create_1_hint_sql_profile.sql) with the GATHER_PLAN_STATISTICS hint. You can use dplan_allstats.sql to see the extended statistics. Here’s an example:

^?View Code NONE

> !sql
sqlplus "/ as sysdba"
 
SQL*Plus: Release 11.2.0.1.0 Production on Thu Jan 28 17:49:56 2010
 
Copyright (c) 1982, 2009, Oracle.  All rights reserved.
 
 
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
 
SYS@LAB112> -- note: SQL_ID 84q0zxfzn5u6s is the statement I want to look at
SYS@LAB112>
SYS@LAB112> @create_1_hint_sql_profile
Enter value for sql_id: 84q0zxfzn5u6s
Enter value for profile_name (PROFILE_sqlid_MANUAL): PROFILE_84q0zxfzn5u6s_GPS
Enter value for category (DEFAULT): 
Enter value for force_matching (false): 
Enter value for hint: GATHER_PLAN_STATISTICS
Profile PROFILE_84q0zxfzn5u6s_MANUAL created.
 
PL/SQL procedure successfully completed.
 
SYS@LAB112> @sql_profiles
Enter value for sql_text: 
Enter value for name: 
 
NAME                           CATEGORY        STATUS   SQL_TEXT                                                               FOR
------------------------------ --------------- -------- ---------------------------------------------------------------------- ---
PROFILE_fgn6qzrvrjgnz          DEFAULT         DISABLED select /*+ index(a SKEW_COL1) */ avg(pk_col) from kso.skew a           NO
PROFILE_69k5bhm12sz98          DEFAULT         DISABLED SELECT dbin.instance_number,        dbin.db_name, dbin.instance_name,  NO
PROFILE_8js5bhfc668rp          DEFAULT         DISABLED select /*+ index(a SKEW_COL2_COL1) */ avg(pk_col) from kso.skew a wher NO
PROFILE_bxd77v75nynd8          DEFAULT         DISABLED select /*+ parallel (a 4) */ avg(pk_col) from kso.skew a where col1 >  NO
PROFILE_8hjn3vxrykmpf          DEFAULT         DISABLED select /*+ invalid_hint (doda) */ avg(pk_col) from kso.skew where col1 NO
PROFILE_7ng34ruy5awxq          DEFAULT         DISABLED select i.obj#,i.ts#,i.file#,i.block#,i.intcols,i.type#,i.flags,i.prope NO
PROFILE_84q0zxfzn5u6s_GPS      DEFAULT         ENABLED  select avg(pk_col) from kso.skew                                       NO
 
7 rows selected.
 
SYS@LAB112> @sql_profile_hints
Enter value for profile_name: PROFILE_84q0zxfzn5u6s_GPS
 
HINT
------------------------------------------------------------------------------------------------------------------------------------------------------
GATHER_PLAN_STATISTICS
 
1 rows selected.
 
SYS@LAB112> @avgskewi
 
AVG(PK_COL)
-----------
   15636133
 
1 row selected.
 
SYS@LAB112> @dplan_allstats
Enter value for sql_id: 84q0zxfzn5u6s
Enter value for child_no: 
 
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  84q0zxfzn5u6s, child number 0
-------------------------------------
select avg(pk_col) from kso.skew where col1 = 136133
 
Plan hash value: 3723858078
 
----------------------------------------------------------------------------------------------------
| Id  | Operation                    | Name      | Starts | E-Rows | A-Rows |   A-Time   | Buffers |
----------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT             |           |      1 |        |      1 |00:00:00.01 |      35 |
|   1 |  SORT AGGREGATE              |           |      1 |      1 |      1 |00:00:00.01 |      35 |
|   2 |   TABLE ACCESS BY INDEX ROWID| SKEW      |      1 |     35 |     32 |00:00:00.01 |      35 |
|*  3 |    INDEX RANGE SCAN          | SKEW_COL1 |      1 |     35 |     32 |00:00:00.01 |       3 |
----------------------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   3 - access("COL1"=136133)
 
Note
-----
   - SQL profile PROFILE_84q0zxfzn5u6s_GPS used for this statement
   - SQL plan baseline SQL_84Q0ZXFZN5U6S_3723858078 used for this statement
 
 
25 rows selected.
 
SYS@LAB112> @sql_hints
Enter value for sql_id: 84q0zxfzn5u6s
Enter value for child_no: 0
 
OUTLINE_HINTS
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
IGNORE_OPTIM_EMBEDDED_HINTS
OPTIMIZER_FEATURES_ENABLE('11.2.0.1')
DB_VERSION('11.2.0.1')
ALL_ROWS
OUTLINE_LEAF(@"SEL$1")
INDEX_RS_ASC(@"SEL$1" "SKEW"@"SEL$1" ("SKEW"."COL1"))
 
6 rows selected.

Works great! Now you don’t have to do all that work just to see the rowsource execution stats.

Note that the Outline Hints stored in the OTHER_XML column of V$sql_plan have no record of the GATHER_PLAN_STATISTICS hint, even though it was clearly executed (otherwise we’d have gotten a warning message in the XPLAN output and no A-Rows column).

Note also that this example was done on an 11gR2 database and that the statement was found in the SQL Management Base (i.e. there was a Baseline in place for this statement already). The Notes section of the plan output shows that both the Profile and the Baseline were used. This got me wondering if it really merged the hints from the Profile and the Baseline. The short answer is YES, it appears that it does. I’ve got an example, but since that’s a different topic and this post is already pretty long, I’ll just save it for another post.

As always, your comments are welcomed.

UPDATE: This has proved pretty useful, so I wrote a little script that just prompts for a sql_id and creates a SQL Profile with the GATHER_PLAN_STATISTICS hint. I have just uploaded it to the site here: gps.sql

SQL Tuning Advisor Profiles

osborne — Wed, 03 Feb 2010 16:05:36 +0000

Warning! While the details of the other points in this post remain valid, the main conclusion of the post (i.e. that the Tuning Advisor can create Profiles with hints other than the “statistics modification” type, opt_estimate for example) is not valid. At this point I have no proof that the Tuning Advisor creates Profiles with the more directive type hints. The conclusion was based on a faulty script (sql_profile_hints) which in certain situations combined hints from multiple Profiles. (The script has been modified now)

Be sure and see the comments from Jonathan and my response for further information. I will leave the remainder of the post unchanged for reference.

============================================================

This is the 2nd post in response to a dialog with Jonathan Lewis regarding SQL Profiles. Here’s a link to the 1st post and a link to the original dialog.

Just a quick recap: Jonathan had pointed out that I was using SQL Profiles in a manner that was not intended by the developers (with which I had to agree). SQL Profiles are created by the SQL Tuning Advisor (STA) primarily to address inaccurate calculations by the optimizer. The first post dealt with the question of whether SQL Profiles were capable of dealing with any/all hints or just the ones most commonly used by STA generated Profiles. This post focuses on a second question, namely whether STA generated Profiles ever had any thing other than the “statistics modification” type hints (opt_estimate, column_stats, table_stats, index_stats, etc…).

So in an effort to answer that question, I did a little looking around on some systems that had implemented a fair number of STA generated Profiles. In order to speed my “looking around”, I wrote a script to do a simple grouping of hints (sql_profile_distinct_hints.sql). <- got to use a little regex in that one ;)

You'll also need my sql_profile_hints.sql script if you want to follow along in your hymnal.

Also note that STA creates Profiles that are named with this format: SYS_SQLPROF_xxxxxxx, so we’ll only look at Profiles that are named in that fashion.

^?View Code NONE

===============
System 1
===============
 
SQL> set echo on
SQL> @sql_profiles
SQL> col category for a15
SQL> col sql_text for a70 trunc
SQL> select name, category, status, sql_text
  2  from dba_sql_profiles
  3  where sql_text like nvl('&sql_text',sql_text)
  4  and name like nvl('&name',name)
  5  order by last_modified
  6  /
Enter value for sql_text: 
Enter value for name: SYS_SQLPROF%
 
NAME                           CATEGORY        STATUS   SQL_TEXT                                                               
------------------------------ --------------- -------- ----------------------------------------------------------------------
SYS_SQLPROF_014715f1c62ec002   DEFAULT         ENABLED  SELECT /*+ OPAQUE_TRANSFORM */ ...
SYS_SQLPROF_0148f3819bdb0003   DEFAULT         ENABLED  SELECT ...
SYS_SQLPROF_0148f3819bd1c002   DEFAULT         ENABLED  SELECT ...
SYS_SQLPROF_0148f3819be38004   DEFAULT         ENABLED  SELECT ...
SYS_SQLPROF_0148f3b92f920008   DEFAULT         ENABLED  SELECT ...
SYS_SQLPROF_0148f3b92f5c4005   DEFAULT         ENABLED  DELETE ...
SYS_SQLPROF_0148f3b92f848007   DEFAULT         ENABLED  SELECT ...
SYS_SQLPROF_014914d83003400b   DEFAULT         ENABLED  SELECT ...
SYS_SQLPROF_014914d8682e800c   DEFAULT         ENABLED  SELECT ...
SYS_SQLPROF_01491c90b1dd0010   DEFAULT         ENABLED  SELECT ...
SYS_SQLPROF_01491dfb1c85c012   DEFAULT         ENABLED  SELECT ...
SYS_SQLPROF_014913e28eec0006   DEFAULT         DISABLED SELECT ... 
SYS_SQLPROF_0148f3b92fa04009   DEFAULT         DISABLED SELECT ...
SYS_SQLPROF_01491ccd37348011   DEFAULT         DISABLED SELECT ...
 
14 rows selected.
 
SQL> 
SQL> @sql_profile_distinct_hints
SQL> set lines 155
SQL> col hint for a50 trunc
SQL> select hint, count(*) from (
  2  select regexp_replace(attr_val,'\(.*$') hint -- eliminate from ( to end of line
  3  from dba_sql_profiles p, sqlprof$attr h
  4  where p.signature = h.signature
  5  and name like ('&profile_name')
  6  )
  7  group by hint
  8  order by hint
  9  /
Enter value for profile_name: SYS_SQLPROF%
 
HINT                                                 COUNT(*)
-------------------------------------------------- ----------
COLUMN_STATS                                               13
FIRST_ROWS                                                  1
IGNORE_OPTIM_EMBEDDED_HINTS                                 1
INDEX_STATS                                                 1
OPTIMIZER_FEATURES_ENABLE                                  14
OPT_ESTIMATE                                              178
TABLE_STATS                                                 2
 
7 rows selected.
 
===============
System 2
===============
 
SQL> @sql_profiles
Enter value for sql_text: 
Enter value for name: 
 
NAME                           CATEGORY        STATUS   SQL_TEXT                                                               
------------------------------ --------------- -------- ---------------------------------------------------------------------- 
SYS_SQLPROF_0149380562710001   DEFAULT         ENABLED  SELECT DISTINCT ...
SYS_SQLPROF_01494247f1d7c007   DEFAULT         ENABLED        INSERT INTO ...
SYS_SQLPROF_01494247f254c008   DEFAULT         ENABLED        INSERT INTO ...
SYS_SQLPROF_01494db7d72f0000   DEFAULT         ENABLED        INSERT INTO ...
PROFILE_1zywf8dtd9a6m          DEFAULT         ENABLED  select ...
PROFILE_as2u1cbquf6t3_moved    DEFAULT         ENABLED  SELECT ...
PROFILE_ay0yn9xczhqk2          DEFAULT         ENABLED  select ...
PROFILE_402tfpm6851du_moved    DEFAULT         ENABLED  SELECT /*+ORDERED ...
PROFILE_0bbnbjt6r17zg          DEFAULT         ENABLED  select /* test 1qkytgsyhfrn6_11.sql */ ...
PROFILE_7kj5cc04njm64          DEFAULT         ENABLED  select /* test 1qkytgsyhfrn6_4a.sql */ ...
PROFILE_1qkytgsyhfrn6_moved    DEFAULT         ENABLED  SELECT /*+ORDERED ...
PROFILE_8xva1napnu034_moved    DEFAULT         ENABLED  SELECT /*+ORDERED USE_NL ...
PROFILE_6njq8vtawcq1m          DEFAULT         ENABLED  DELETE FROM ...
 
13 rows selected.
 
SQL> 
SQL> @sql_profile_distinct_hints
Enter value for profile_name: SYS_SQLPROF%
 
HINT                                                 COUNT(*)
-------------------------------------------------- ----------
COLUMN_STATS                                                3
IGNORE_OPTIM_EMBEDDED_HINTS                                 3
INDEX_STATS                                                 2
OPTIMIZER_FEATURES_ENABLE                                   4
OPT_ESTIMATE                                              416
TABLE_STATS                                                 1
 
6 rows selected.
 
===============
System 3
===============
 
 
SQL> @sql_profiles
Enter value for sql_text:  
Enter value for name: SYS%
 
NAME                           CATEGORY        STATUS   SQL_TEXT                                                               FOR
------------------------------ --------------- -------- ---------------------------------------------------------------------- ---
SYS_SQLPROF_0146d1cc64d5c001   DEFAULT         ENABLED  select ...
SYS_SQLPROF_0146d1ce58150002   DEFAULT         ENABLED  select ...
SYS_SQLPROF_0146d1cfd2488003   DEFAULT         ENABLED  select ...
SYS_SQLPROF_0146d1e45f1f0005   DEFAULT         ENABLED  select ...
SYS_SQLPROF_0146f4d7bee40007   DEFAULT         ENABLED  select ...
SYS_SQLPROF_014738be1cb64000   DEFAULT         ENABLED  select ...
SYS_SQLPROF_0148f25055a28004   DEFAULT         ENABLED  SELECT ...
SYS_SQLPROF_0148f2528b754005   DEFAULT         ENABLED  select ...
SYS_SQLPROF_0148f252e7e84007   DEFAULT         ENABLED  SELECT DISTINCT ...
SYS_SQLPROF_0148f2553429800a   DEFAULT         ENABLED  SELECT /*+ FULL (...
SYS_SQLPROF_0248f26958210000   DEFAULT         ENABLED  select * from ...
SYS_SQLPROF_0148f37413ef000c   DEFAULT         ENABLED  SELECT ...
SYS_SQLPROF_0148fcc1fa89c001   DEFAULT         ENABLED  SELECT ...
SYS_SQLPROF_014909a2d7bc4002   DEFAULT         ENABLED  select ...
SYS_SQLPROF_01491c971e1e4003   DEFAULT         ENABLED  select ...
SYS_SQLPROF_01491ce0f3c1800b   DEFAULT         ENABLED  SELECT ...
SYS_SQLPROF_01491ce50f9e800c   DEFAULT         ENABLED  SELECT ...
SYS_SQLPROF_01491f66feb0400d   DEFAULT         ENABLED  SELECT ...
SYS_SQLPROF_02492723d3778000   SAVED           DISABLED select ...
SYS_SQLPROF_01491ce04c38c008   DEFAULT         DISABLED SELECT ...
 
20 rows selected.
 
SQL> @sql_profile_distinct_hints
Enter value for profile_name: SYS_SQLPROF%
 
HINT                                                 COUNT(*)
-------------------------------------------------- ----------
ALL_ROWS                                                    3
COLUMN_STATS                                               12
FIRST_ROWS                                                  1
FULL                                                        3
IGNORE_OPTIM_EMBEDDED_HINTS                                 2
INDEX                                                       2
INDEX_RS_ASC                                                2
INDEX_STATS                                                12
LEADING                                                     1
OPTIMIZER_FEATURES_ENABLE                                   6
OPT_ESTIMATE                                              212
OPT_PARAM                                                   2
OUTLINE                                                     2
OUTLINE_LEAF                                                1
SWAP_JOIN_INPUTS                                            1
TABLE_STATS                                                 4
UNNEST                                                      1
USE_HASH                                                    2
USE_NL                                                      4
 
19 rows selected.
 
SQL> -- different
SQL> -- After a little looking I found that one Profile had all the non standard hints
SQL>
SQL> @sql_profile_hints
Enter value for profile_name: SYS_SQLPROF_02492723d3778000
 
HINT
------------------------------------------------------------------------------------------------------------------------------------------------------
ALL_ROWS
ALL_ROWS
FULL(@"SEL$5DA710D3" "A"@"SEL$1")
FULL(@"SEL$5DA710D3" "B"@"SEL$1")
FULL(@"SEL$5DA710D3" "TAB_XXX1"@"SEL$2")
IGNORE_OPTIM_EMBEDDED_HINTS
INDEX(@"SEL$5DA710D3" "D"@"SEL$1" ("TAB_XXX3"."STRATEGY_ID" "TAB_XXX3"."FF_STATUS"))
INDEX(@"SEL$5DA710D3" "F"@"SEL$1" ("TAB_XXX2"."COL1" "TAB_XXX2"."COL2" "TAB_XXX2"."COL3" "TAB_XXX2"."COL4"))
INDEX_RS_ASC(@"SEL$5DA710D3" "C"@"SEL$1" ("TAB_XXX3"."COL1" "TAB_XXX3"."COL2" "TAB_XXX3"."COL3"))
INDEX_RS_ASC(@"SEL$5DA710D3" "E"@"SEL$1" ("TAB_XXX4"."COL1" "TAB_XXX4"."COL2" "TAB_XXX4"."COL3"))
LEADING(@"SEL$5DA710D3" "A"@"SEL$1" "B"@"SEL$1" "TAB_XXX1"@"SEL$2" "F"@"SEL$1" "C"@"SEL$1" "E"@"SEL$1" "D"@"SEL$1")
OPTIMIZER_FEATURES_ENABLE('10.2.0.4')
OPT_ESTIMATE(@"SEL$5DA710D3", INDEX_FILTER, "F"@"SEL$1", IDX$$_1AA260002, SCALE_ROWS=8.883203639e-06)
OPT_ESTIMATE(@"SEL$5DA710D3", INDEX_SKIP_SCAN, "F"@"SEL$1", IDX$$_1AA260002, SCALE_ROWS=8.883203639e-06)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("B"@"SEL$1", "A"@"SEL$1"), SCALE_ROWS=4.446153275)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("C"@"SEL$1", "A"@"SEL$1"), SCALE_ROWS=7.884506683)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("D"@"SEL$1", "C"@"SEL$1", "A"@"SEL$1", "TAB_XXX1"@"SEL$2"), SCALE_ROWS=1.308307653)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "A"@"SEL$1"), SCALE_ROWS=25.60960842)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "B"@"SEL$1", "A"@"SEL$1", "TAB_XXX1"@"SEL$2"), SCALE_ROWS=135.982493)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "C"@"SEL$1", "A"@"SEL$1"), SCALE_ROWS=190.2944942)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "C"@"SEL$1", "A"@"SEL$1", "TAB_XXX1"@"SEL$2"), SCALE_ROWS=26.52093258)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "D"@"SEL$1", "C"@"SEL$1", "A"@"SEL$1"), SCALE_ROWS=259.4309108)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "D"@"SEL$1", "C"@"SEL$1", "B"@"SEL$1", "A"@"SEL$1", "TAB_XXX1"@"SEL$2"), SCALE_ROWS=862.935946
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("F"@"SEL$1", "B"@"SEL$1"), SCALE_ROWS=26.34181566)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("F"@"SEL$1", "B"@"SEL$1", "A"@"SEL$1"), SCALE_ROWS=839.9683673)
OPT_ESTIMATE(@"SEL$5DA710D3", TABLE, "D"@"SEL$1", SCALE_ROWS=11.39782103)
OPT_PARAM('optimizer_index_caching' 60)
OPT_PARAM('optimizer_index_cost_adj' 80)
OUTLINE(@"SEL$1")
OUTLINE(@"SEL$2")
OUTLINE_LEAF(@"SEL$5DA710D3")
SWAP_JOIN_INPUTS(@"SEL$5DA710D3" "TAB_XXX1"@"SEL$2")
UNNEST(@"SEL$2")
USE_HASH(@"SEL$5DA710D3" "B"@"SEL$1")
USE_HASH(@"SEL$5DA710D3" "TAB_XXX1"@"SEL$2")
USE_NL(@"SEL$5DA710D3" "C"@"SEL$1")
USE_NL(@"SEL$5DA710D3" "D"@"SEL$1")
USE_NL(@"SEL$5DA710D3" "E"@"SEL$1")
USE_NL(@"SEL$5DA710D3" "F"@"SEL$1")
 
39 rows selected.
 
SQL> -- So did someone create a manual Profile and name it SYS_SQLPROF_xxxxx ?
SQL> -- Seems unlikely
SQL> -- Let's try to reproduce this behavior (it's on sql_id: 425p8m2qdxdtm)
SQL>
SQL> @create_tuning_task
Task_Name: KSO TEST 425p8m2qdxdtm
Enter value for sql_id: 425p8m2qdxdtm
Enter value for time_limit: 3000
 
PL/SQL procedure successfully completed.
 
 
RECOMMENDATIONS
------------------------------------------------------------------------------------------------------------------------------------------------------
GENERAL INFORMATION SECTION
-------------------------------------------------------------------------------
Tuning Task Name                  : KSO TEST 425p8m2qdxdtm
Tuning Task Owner                 : SYS
Scope                             : COMPREHENSIVE
Time Limit(seconds)               : 3000
Completion Status                 : COMPLETED
Started at                        : 01/26/2010 15:25:08
Completed at                      : 01/26/2010 15:35:05
Number of SQL Profile Findings    : 1
 
-------------------------------------------------------------------------------
Schema Name: SCHEMA_XXX
SQL ID     : 425p8m2qdxdtm
SQL Text   : select ...
 
-------------------------------------------------------------------------------
FINDINGS SECTION (1 finding)
-------------------------------------------------------------------------------
 
1- SQL Profile Finding (see explain plans section below)
--------------------------------------------------------
  A potentially better execution plan was found for this statement.
 
  Recommendation (estimated benefit: 73.58%)
  ------------------------------------------
  - Consider accepting the recommended SQL profile.
    execute dbms_sqltune.accept_sql_profile(task_name => 'KSO TEST
            425p8m2qdxdtm', replace => TRUE);
 
-------------------------------------------------------------------------------
ADDITIONAL INFORMATION SECTION
-------------------------------------------------------------------------------
- SQL Profile "PROFILE_425p8m2qdxdtm" exists for this statement and was
  ignored during the tuning process.
 
-------------------------------------------------------------------------------
EXPLAIN PLANS SECTION
-------------------------------------------------------------------------------
 
1- Original With Adjusted Cost
------------------------------
Plan hash value: 3884938694
 
--------------------------------------------------------------------------------------------------------------------
| Id  | Operation                       | Name                     | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |
--------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                |                          |     1 |   207 |       |   171K  (2)| 00:34:18 |
|   1 |  HASH UNIQUE                    |                          |     1 |   207 |       |   171K  (2)| 00:34:18 |
|   2 |   NESTED LOOPS                  |                          |     1 |   207 |       |   171K  (2)| 00:34:18 |
|   3 |    NESTED LOOPS                 |                          |     1 |   198 |       |   171K  (2)| 00:34:18 |
|   4 |     NESTED LOOPS                |                          |     7 |  1092 |       |   171K  (2)| 00:34:18 |
|   5 |      NESTED LOOPS               |                          |    69 |  9039 |       |   171K  (2)| 00:34:16 |
|*  6 |       HASH JOIN RIGHT ANTI      |                          | 11209 |   996K|  5912K|   171K  (2)| 00:34:16 |
|...
 
--------------------------------------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   6 - access(TAB_XXX1.COL1)
   ...
 
2- Using SQL Profile
--------------------
Plan hash value: 3101771407
 
----------------------------------------------------------------------------------------------------------------------
| Id  | Operation                         | Name                     | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |
----------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                  |                          |     1 |   207 |       | 45293   (2)| 00:09:04 |
|   1 |  HASH UNIQUE                      |                          |     1 |   207 |       | 45293   (2)| 00:09:04 |
|   2 |   NESTED LOOPS ANTI               |                          |     1 |   207 |       | 45292   (2)| 00:09:04 |
|   3 |    NESTED LOOPS                   |                          |     1 |   170 |       | 45290   (2)| 00:09:04 |
|   4 |     NESTED LOOPS                  |                          |   122 | 15860 |       | 45289   (2)| 00:09:04 |
|*  5 |      HASH JOIN                    |                          |  2868 |   277K|  4480K| 40696   (2)| 00:08:09 |
|*  6 |       HASH JOIN                   |                          | 66442 |  3698K|    22M| 36124   (2)| 00:07:14 |
| ...
 
----------------------------------------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   5 - access(TAB_XXX1.COL1)
   ...
 
-------------------------------------------------------------------------------
 
 
SQL> set echo on
SQL> set echo off
SQL> !cat accept_sql_profile.sql
exec dbms_sqltune.accept_sql_profile(task_name => '&task_name',category => '&category');
 
SQL> @accept_sql_profile
Enter value for task_name: KSO TEST 425p8m2qdxdtm
Enter value for category: TEST
 
PL/SQL procedure successfully completed.
 
SQL>  select name, category, status, sql_text
  2  from dba_sql_profiles
  3  where sql_text like nvl('&sql_text',sql_text)
  4  and name like nvl('&name',name)
  5  and categeory like 'TEST'
  6* order by last_modified;
Enter value for sql_text: 
Enter value for name: 
 
NAME                           CATEGORY        STATUS   SQL_TEXT                                                               
------------------------------ --------------- -------- ---------------------------------------------------------------------- 
SYS_SQLPROF_01499b27a4ea4000   TEST            ENABLED  select ...
 
 
SQL> @sql_profile_hints
Enter value for profile_name: SYS_SQLPROF_01499b27a4ea4000
 
HINT
------------------------------------------------------------------------------------------------------------------------------------------------------
ALL_ROWS
ALL_ROWS
ALL_ROWS
FULL(@"SEL$5DA710D3" "A"@"SEL$1")
FULL(@"SEL$5DA710D3" "B"@"SEL$1")
FULL(@"SEL$5DA710D3" "TAB_XXX1"@"SEL$2")
IGNORE_OPTIM_EMBEDDED_HINTS
INDEX(@"SEL$5DA710D3" "D"@"SEL$1" ("TAB_XXX2"."COL1" "TAB_XXX2"."COL2"))
INDEX(@"SEL$5DA710D3" "F"@"SEL$1" ("TAB_XXX3"."COL1" "TAB_XXX3"."COL2" "TAB_XXX3"."COL3"))
INDEX_RS_ASC(@"SEL$5DA710D3" "C"@"SEL$1" ("TAB_XXX2"."COL1" "TAB_XXX2"."COL2" "TAB_XXX2"."COL3"))
INDEX_RS_ASC(@"SEL$5DA710D3" "E"@"SEL$1" ("TAB_XXX3"."COL1" "TAB_XXX3"."COL2" "TAB_XXX3"."COL3"))
LEADING(@"SEL$5DA710D3" "A"@"SEL$1" "B"@"SEL$1" "TAB_XXX1"@"SEL$2" "F"@"SEL$1" "C"@"SEL$1" "E"@"SEL$1" "D"@"SEL$1")
OPTIMIZER_FEATURES_ENABLE('10.2.0.4')
OPT_ESTIMATE(@"SEL$5DA710D3", INDEX_FILTER, "F"@"SEL$1", IDX$$_1AA260002, SCALE_ROWS=8.883203639e-06)
OPT_ESTIMATE(@"SEL$5DA710D3", INDEX_FILTER, "F"@"SEL$1", TAB_XXX3_IX1, SCALE_ROWS=6.531081416e-06)
OPT_ESTIMATE(@"SEL$5DA710D3", INDEX_SKIP_SCAN, "A"@"SEL$1", TAB_XXX2_IX3, SCALE_ROWS=1.481282096e-06)
OPT_ESTIMATE(@"SEL$5DA710D3", INDEX_SKIP_SCAN, "A"@"SEL$1", TAB_XXX2_IX7, SCALE_ROWS=1.481282096e-06)
OPT_ESTIMATE(@"SEL$5DA710D3", INDEX_SKIP_SCAN, "F"@"SEL$1", IDX$$_1AA260002, SCALE_ROWS=8.883203639e-06)
OPT_ESTIMATE(@"SEL$5DA710D3", INDEX_SKIP_SCAN, "F"@"SEL$1", TAB_XXX_IX1, SCALE_ROWS=6.531081416e-06)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("B"@"SEL$1", "A"@"SEL$1"), SCALE_ROWS=4.446153275)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("C"@"SEL$1", "A"@"SEL$1"), SCALE_ROWS=7.884506683)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("C"@"SEL$1", "A"@"SEL$1"), SCALE_ROWS=8.321869785)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("D"@"SEL$1", "C"@"SEL$1", "A"@"SEL$1", "TAB_XXX1"@"SEL$2"), SCALE_ROWS=1.308307653)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "A"@"SEL$1"), SCALE_ROWS=22.98505769)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "A"@"SEL$1"), SCALE_ROWS=25.60960842)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "A"@"SEL$1", "TAB_XXX1"@"SEL$2"), SCALE_ROWS=4.872324946)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "B"@"SEL$1", "A"@"SEL$1", "TAB_XXX1"@"SEL$2"), SCALE_ROWS=135.982493)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "C"@"SEL$1", "A"@"SEL$1"), SCALE_ROWS=188.6083111)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "C"@"SEL$1", "A"@"SEL$1"), SCALE_ROWS=190.2944942)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "C"@"SEL$1", "A"@"SEL$1", "TAB_XXX1"@"SEL$2"), SCALE_ROWS=26.52093258)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "C"@"SEL$1", "A"@"SEL$1", "TAB_XXX1"@"SEL$2"), SCALE_ROWS=40.52912454)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "D"@"SEL$1", "C"@"SEL$1", "A"@"SEL$1"), SCALE_ROWS=259.4309108)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "D"@"SEL$1", "C"@"SEL$1", "A"@"SEL$1", "TAB_XXX1"@"SEL$2"), SCALE_ROWS=41.60863798)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "D"@"SEL$1", "C"@"SEL$1", "B"@"SEL$1", "A"@"SEL$1", "TAB_XXX1"@"SEL$2"), SCALE_ROWS=1102.68181
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("E"@"SEL$1", "D"@"SEL$1", "C"@"SEL$1", "B"@"SEL$1", "A"@"SEL$1", "TAB_XXX1"@"SEL$2"), SCALE_ROWS=862.935946
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("F"@"SEL$1", "B"@"SEL$1"), SCALE_ROWS=26.34181566)
OPT_ESTIMATE(@"SEL$5DA710D3", JOIN, ("F"@"SEL$1", "B"@"SEL$1", "A"@"SEL$1"), SCALE_ROWS=839.9683673)
OPT_ESTIMATE(@"SEL$5DA710D3", TABLE, "D"@"SEL$1", SCALE_ROWS=11.39782103)
OPT_ESTIMATE(@"SEL$5DA710D3", TABLE, "D"@"SEL$1", SCALE_ROWS=11.95241429)
OPT_PARAM('optimizer_index_caching' 61)
OPT_PARAM('optimizer_index_cost_adj' 79)
OUTLINE(@"SEL$1")
OUTLINE(@"SEL$2")
OUTLINE_LEAF(@"SEL$5DA710D3")
SWAP_JOIN_INPUTS(@"SEL$5DA710D3" "TAB_XXX1"@"SEL$2")
UNNEST(@"SEL$2")
USE_HASH(@"SEL$5DA710D3" "B"@"SEL$1")
USE_HASH(@"SEL$5DA710D3" "TAB_XXX1"@"SEL$2")
USE_NL(@"SEL$5DA710D3" "C"@"SEL$1")
USE_NL(@"SEL$5DA710D3" "D"@"SEL$1")
USE_NL(@"SEL$5DA710D3" "E"@"SEL$1")
USE_NL(@"SEL$5DA710D3" "F"@"SEL$1")
 
52 rows selected.
 
SQL> -- so STA really did create a Profile with all these hints, interesting...

A couple of things worth noting:

OPT_ESTIMATE was far and away the most popular hint. This is followed by the occasional XXX_STATS hints. The OPTIMIZER_FEATURES_ENABLE hint shows up in almost all the STA Profiles. Also, the IGNORE_OPTIM_EMBEDDED_HINTS shows up fairly regularly, but not always (not sure why it’s there sometimes and not others - I thought it might show up only when there are actually embedded hints in the SQL text, but that is not the case). Occasionally, an ALL_ROWS or FIRST_ROWS hint will show up. And finally, in rare cases, a full blown Profile with many directives will show up as in the one case I was able to reproduce. I really have no idea when/why this kicks in. But it doesn’t seem to happen very often. However, it does give a strong clue that most, if not all, valid hints can be applied by SQL Profiles.

And a couple of editorial points:

1. I don’t really like STA generated Profiles as a long term solution. Since they primarily apply fudge factors (via the opt_estimate hint) to the optimizer calculations (which are based in large part on statistics), they tend to go bad fairly quickly as the statistics change.

2. I think they are very useful though for finding where the optimizer has gone wrong. Anywhere there is an opt_estimate hint with a really large (or small) scaling factor, the optimizer had some kind of problem. This could be due to bad stats, or just a short coming in the optimizer itself. But it can be a very good pointer to why/where the optimizer is not doing a good job.

3. There is no reason you can’t use an STA Profile to get a decent plan and then do something else to make it more permanent - i.e. create an Outline or Baseline or even another Profile with all the hints necessary to reproduce the plan. This is of course if your intent is to “lock” the plan.

4. If you have a short coming in the optimizer (such as 10g’s lack of knowledge about correlated columns) then the opt_estimate hint in the STA Profiles may be a viable long term solution. Of course, this implies that you know enough about how the calculations are done to be sure that a certain calculation will always be off by a calculatable (did I just make that word up?) factor.

As always, your comments are welcomed.

Single Hint SQL Profiles

osborne — Mon, 01 Feb 2010 21:25:22 +0000

Seems like all I ever write about these days is SQL Profiles. I do other stuff, honest! It just seems like getting Oracle to do what you want when you can’t touch the code is the closest thing to “Magic” that DBAs get to do. By the way, software developers get to create “Magic” all the time. They have the ability to create something out of nothing. I think it’s one of the coolest jobs ever. Of course a lot of art is like that too. But painters and sculptors start with some sort of raw material - programmers don’t. Musicians don’t really use raw material either, but music is much less tangible than software. Maybe one of these days I’ll write a little on the similarities between musicians and software developers, but right now I’m way off in the weeds. Back to the subject at hand.

A few weeks ago Jonathan Lewis called me to task on a couple of posts regarding SQL Profiles (in a very nice collegial sort of way). You can see the original dialog here. One of his main points was that SQL Profiles were not meant to be a generic mechanism for forcing a particular execution plan the way Outlines are. There is after all, no documented way (that I’m aware of) to directly create a SQL Profile on a statement. I had to agree with him that I was using them in a way that was not necessarily intended. Outlines were designed to lock execution plans, SQL Profiles were designed to overcome shortcomings in the optimizer with regards to statistics. Tom Kyte described SQL Profiles like this:

So, a SQL profile is sort of like gathering statistics on A QUERY - which involves many
tables, columns and the like….

In fact - it is just like gathering statistics for a query, it stores additional
information in the dictionary which the optimizer uses at optimization time to determine
the correct plan. The SQL Profile is not “locking a plan in place”, but rather giving
the optimizer yet more bits of information it can use to get the right plan.

Tom is referring to the documented way of creating a SQL Profile which is to use the SQL Tuning Advisor. The Tuning Advisor verifies the optimizer’s calculations and can create a SQL Profile that corrects the calculations, if they are found to be in error. The corrections most often come in the form of OPT_ESTIMATE hints which apply a scaling factor at various places in the optimizer’s calculations. But take note that the underlying mechanism of applying these corrections is hints. So a SQL Profile is ultimately a mechanism for applying a set of stored hints to a SQL statement (or set of statements) behind the scenes. And whether it was intended by the developers or not, this gives us a tremendously powerful tool. With this tool we can influence (and often times control) execution plans for statements coming from application code that is difficult or impossible to change.

By the way, all this discussion of SQL Profiles and whether they should be used as a generic mechanism for affecting plans without touching a SQL statement is probably going to be fairly short lived. SQL Baselines (introduced in 11g) are the latest revision of the “behind the scenes hint application” idea. They will most likely make SQL Profiles a less attractive option in the future. Fortunately, it is a simple matter to convert a SQL Profile into a Baseline. And SQL Profiles still work fine in 11g as well (so do Outlines for that matter), but on the off chance that Oracle decides to do away with SQL Profiles (or alter their behavior) in some future release, it’s comforting to know that we are not headed down a dead end street.

Back to the subject at hand. Jonathan expressed concern and doubt on a couple of points:

That Profiles really were a generic mechanism to apply any hint behind the scenes. (i.e. that they could be used to apply any arbitrary hint, not just the hints created by the Tuning Advisor - like opt_estimate).
Whether Profiles created by the Tuning Advisor ever had hints other than “Change the Optimizer Calculations” type hints (i.e. opt_estimate, index_stats, table_stats, etc…).

I think we finally agreed that they can be used to apply hints in a generic fashion, even if that was not the intention of the developers. Nevertheless, here is a simple test case to demonstrate that they can be used for that purpose. I created a little script to create a SQL Profile with a single manually typed hint called create_1_hint_sql_profile.sql. Please note that the syntax can be quite finicky with regards to query block names and aliases.

^?View Code NONE

> !sql
sqlplus "/ as sysdba"
 
SQL*Plus: Release 11.2.0.1.0 Production on Mon Jan 25 15:23:02 2010
 
Copyright (c) 1982, 2009, Oracle.  All rights reserved.
 
 
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
 
SYS@LAB112> @flush_pool
 
System altered.
 
SYS@LAB112> @avgskewi
 
AVG(PK_COL)
-----------
   15636133
 
SYS@LAB112> @find_sql
Enter value for sql_text: %skew%
Enter value for sql_id: 
 
SQL_ID         CHILD  PLAN_HASH      EXECS     AVG_ETIME      AVG_LIO SQL_TEXT
------------- ------ ---------- ---------- ------------- ------------ ------------------------------------------------------------
84q0zxfzn5u6s      0 3723858078          1           .05          190 select avg(pk_col) from kso.skew where col1 = 136133
 
SYS@LAB112> @dplan
Enter value for sql_id: 84q0zxfzn5u6s
Enter value for child_no: 
 
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  84q0zxfzn5u6s, child number 0
-------------------------------------
select avg(pk_col) from kso.skew where col1 = 136133
 
Plan hash value: 3723858078
 
------------------------------------------------------------------------------------------
| Id  | Operation                    | Name      | Rows  | Bytes | Cost (%CPU)| Time     |
------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT             |           |       |       |    35 (100)|          |
|   1 |  SORT AGGREGATE              |           |     1 |    24 |            |          |
|   2 |   TABLE ACCESS BY INDEX ROWID| SKEW      |    35 |   840 |    35   (0)| 00:00:01 |
|*  3 |    INDEX RANGE SCAN          | SKEW_COL1 |    35 |       |     3   (0)| 00:00:01 |
------------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   3 - access("COL1"=136133)
 
 
20 rows selected.
 
SYS@LAB112> -- let's create a 1 hint profile to force a full table scan
SYS@LAB112> 
SYS@LAB112> @create_1_hint_sql_profile
Enter value for sql_id: 84q0zxfzn5u6s
Enter value for profile_name (PROFILE_sqlid_MANUAL): 
Enter value for category (DEFAULT): 
Enter value for force_matching (false): 
Enter value for hint: full(skew)
Profile PROFILE_84q0zxfzn5u6s_MANUAL created.
 
PL/SQL procedure successfully completed.
 
SYS@LAB112> @sql_profiles   
Enter value for sql_text: 
Enter value for name: 
 
NAME                           CATEGORY        STATUS   SQL_TEXT                                                               FOR
------------------------------ --------------- -------- ---------------------------------------------------------------------- ---
PROFILE_fgn6qzrvrjgnz          DEFAULT         DISABLED select /*+ index(a SKEW_COL1) */ avg(pk_col) from kso.skew a           NO
PROFILE_69k5bhm12sz98          DEFAULT         DISABLED SELECT dbin.instance_number,        dbin.db_name, dbin.instance_name,  NO
PROFILE_8js5bhfc668rp          DEFAULT         DISABLED select /*+ index(a SKEW_COL2_COL1) */ avg(pk_col) from kso.skew a wher NO
PROFILE_bxd77v75nynd8          DEFAULT         DISABLED select /*+ parallel (a 4) */ avg(pk_col) from kso.skew a where col1 >  NO
PROFILE_8hjn3vxrykmpf          DEFAULT         DISABLED select /*+ invalid_hint (doda) */ avg(pk_col) from kso.skew where col1 NO
PROFILE_7ng34ruy5awxq          DEFAULT         DISABLED select i.obj#,i.ts#,i.file#,i.block#,i.intcols,i.type#,i.flags,i.prope NO
PROFILE_84q0zxfzn5u6s_MANUAL   DEFAULT         ENABLED  select avg(pk_col) from kso.skew                                       NO
 
7 rows selected.
 
SYS@LAB112> @sql_profile_hints
Enter value for profile_name: PROFILE_84q0zxfzn5u6s_MANUAL
 
HINT
------------------------------------------------------------------------------------------------------------------------------------------------------
full(skew)
 
1 rows selected.
 
SYS@LAB112> @avgskewi
 
AVG(PK_COL)
-----------
   15636133
 
1 row selected.
 
SYS@LAB112> @find_sql
Enter value for sql_text: 
Enter value for sql_id: 84q0zxfzn5u6s
 
SQL_ID         CHILD  PLAN_HASH      EXECS     AVG_ETIME      AVG_LIO SQL_TEXT
------------- ------ ---------- ---------- ------------- ------------ ------------------------------------------------------------
84q0zxfzn5u6s      0 3723858078          1           .01           86 select avg(pk_col) from kso.skew where col1 = 136133
 
1 row selected.
 
SYS@LAB112> @dplan
Enter value for sql_id: 84q0zxfzn5u6s
Enter value for child_no: 0
 
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  84q0zxfzn5u6s, child number 0
-------------------------------------
select avg(pk_col) from kso.skew where col1 = 136133
 
Plan hash value: 3723858078
 
------------------------------------------------------------------------------------------
| Id  | Operation                    | Name      | Rows  | Bytes | Cost (%CPU)| Time     |
------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT             |           |       |       |    35 (100)|          |
|   1 |  SORT AGGREGATE              |           |     1 |    24 |            |          |
|   2 |   TABLE ACCESS BY INDEX ROWID| SKEW      |    35 |   840 |    35   (0)| 00:00:01 |
|*  3 |    INDEX RANGE SCAN          | SKEW_COL1 |    35 |       |     3   (0)| 00:00:01 |
------------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   3 - access("COL1"=136133)
 
Note
-----
   - SQL profile PROFILE_84q0zxfzn5u6s_MANUAL used for this statement
 
 
24 rows selected.
 
SYS@LAB112> -- didn't work - i.e. Profile got created and used, but the hint was ignored
SYS@LAB112> -- oh yeah, Query Block needed
SYS@LAB112> 
SYS@LAB112> @drop_sql_profile
Enter value for profile_name: PROFILE_84q0zxfzn5u6s_MANUAL
 
PL/SQL procedure successfully completed.
 
SYS@LAB112> -- must reload SQL statement for create_1_hint_sql_profile to work
SYS@LAB112> @avgskewi
 
AVG(PK_COL)
-----------
   15636133
 
1 row selected.
 
SYS@LAB112> @create_1_hint_sql_profile
Enter value for sql_id: 84q0zxfzn5u6s
Enter value for profile_name (PROFILE_sqlid_MANUAL): 
Enter value for category (DEFAULT): 
Enter value for force_matching (false): 
Enter value for hint: full( SKEW@SEL1$ )
Profile PROFILE_84q0zxfzn5u6s_MANUAL created.
 
PL/SQL procedure successfully completed.
 
SYS@LAB112> @sql_profile_hints
Enter value for profile_name: PROFILE_84q0zxfzn5u6s_MANUAL
 
HINT
------------------------------------------------------------------------------------------------------------------------------------------------------
full( SKEW@SEL$1 )
 
1 rows selected.
 
SYS@LAB112> @avgskewi
 
 
AVG(PK_COL)
-----------
   15636133
 
1 row selected.
 
SYS@LAB112> @dplan
Enter value for sql_id: 84q0zxfzn5u6s
Enter value for child_no: 
 
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  84q0zxfzn5u6s, child number 0
-------------------------------------
select avg(pk_col) from kso.skew where col1 = 136133
 
Plan hash value: 568322376
 
---------------------------------------------------------------------------
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |       |       | 28360 (100)|          |
|   1 |  SORT AGGREGATE    |      |     1 |    24 |            |          |
|*  2 |   TABLE ACCESS FULL| SKEW |    35 |   840 | 28360   (1)| 00:05:41 |
---------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   2 - filter("COL1"=136133)
 
Note
-----
   - SQL profile PROFILE_84q0zxfzn5u6s_MANUAL used for this statement
 
 
23 rows selected.
 
SYS@LAB112> -- so that worked

Note that the syntax can be tricky requiring correct Query Block name (and aliases if there are any). SEL$1 is the default name for the query block of a simple select. (DEL$1 for deletes, UPD$1 for updates) As you can see from the example, the FULL hint was silently ignored without the QB name.

Since this post has gotten a little long, I’ll save my response to Johnathan’s second concern for the next post. Please let me know if you have any questions.

Another Talk at Hotsos Symposium - 2010

osborne — Thu, 28 Jan 2010 16:35:24 +0000

Looks like I’ll be doing a second presentation at the Hotsos Symposium this year (due to a cancellation by another presenter). The first presentation will be “Controlling Execution Plans (without Touching the Code)”. This talk will be focused on hint based mechanisms (Outlines, SQL Profiles, SQL Baselines). The second topic will be “My Favorite Scripts”. It will be an interactive session demonstrating SQL scripts that I use most often when diagnosing Oracle issues. Here’s a link to the abstracts for my talks (pay no attention to the picture of the person dressed up as a pirate).

I am really excited about this year’s line up of speakers and I highly recommend the event for anyone that is interested in making Oracle perform better. Here’s a link to the signup page. By the way, you can save $100 if you sign up before Feb. 12th. Hope to see you there.

Temp Usage

osborne — Wed, 20 Jan 2010 19:38:15 +0000

There was a post on the oracle-l list the other day regarding disk space usage in the temp tablespace. Rich Jesse posted a neat little script for capturing large consumers of temp space along with the user and SQL statement responsible. I started playing around with the script and noticed that it wasn’t reporting the correct SQL Statement. Turns out there is a bug having to do with the definition of v$sort_usage. It’s apparently been around since 10gR1.

There is a listing on Metalink (sorry,… I mean My Oracle Support). I’ve pasted in the interesting bits here:

Bug 7210183: SQL_ID VALUE IS NOT SAME IN V$TEMPSEG_USAGE AND OTHER VIEWS.
~~~~~~~~~~~
The actual problem here looks to be in the definition of
GV$SORT_USAGE thus:

In RDBMS_MAIN_LINUX_080825:
GV$TEMPSEG_USAGE is a synonym resolving to GV_$SORT_USAGE.

GV_$SORT_USAGE is a VIEW which selects various columns from
GV$SORT_USAGE.

GV$SORT_USAGE has a definition like this:
select x$ktsso.inst_id, username, username, ktssoses, ktssosno,
prev_sql_addr, prev_hash_value, prev_sql_id, <<<<< NOTICE HERE
ktssotsn, ....
from x$ktsso, v$session
where ktssoses = v$session.saddr and ktssosno = v$session.serial#

So GV$SORT_USAGE SQL_ID etc.. are just taken from the sessions
current PREV_* columns. This is not correct. The sort usage
in a session could be tied to any of the open cursors for that
session. The above definition just looks plain wrong.
*** 09/09/08 05:45 pm ***
*** 09/10/08 12:06 am *** (DEL: Impact/Symptom->WRONG RESULTS )
*** 09/10/08 12:06 am *** (ADD: Impact/Symptom->WRONG RESULTS )
*** 09/10/08 12:06 am ***
I am not sure that in all cases using V$SESSION.SQL_ADDRESS
etc.. would be correct.
eg: Consider that the session does something like the
following:
Open cursor 1
Parse , execute and fetch one row from a SQL that needs
temp space in part of the plan
Open cursor 2
select * from dual (no sorting)
At this point V$SESSION.SQL_ID would be for the
select from dual

I do not believe that we actually have the information
available to return the correct SQL_ID etc.. this view.

eg: a. Consider this simple test:
create global temporary table foo ( a number );
insert into foo values(10);
b. In a second session do:
select sql_id from v$sort_usage;
c. Now in the first session issue any old select:
select 99 from dual;
d. And check V$SORT_USAGE:
select sql_id from v$sort_usage;
e. Repeat c and d over using a different SQL each time
in c . eg: select 77 from dual A;

In this example V$SORT_USAGE will show us which session
has the temp space (SESSION_ADDR) but the SQL which
created the temp space is not even available as an open
cursor against that session (the insert created the temp
space).

It looks like this really needs a larger change - something
like capturing the SQL_ID etc.. at the time that the temp seg
gets created and then exposing that information through some
new X$ colums in x$ktsso ?

So I modified the script to report the current sql_id instead.
Here’s a quick replay of the investigation starting with the original query:

^?View Code SQL

SYS@LAB1024> -- original query
SYS@LAB1024> l      
  1  SELECT sysdate "TIME_STAMP", vsu.username, vsu.sql_id, vsu.tablespace,
  2  vsu.usage_mb, vst.sql_text, vp.spid
  3          FROM
  4          (
  5                  SELECT username, sqladdr, sqlhash, sql_id, tablespace, session_addr,
  6  sum(blocks)*8192/1024/1024 "USAGE_MB"
  7                  FROM v$sort_usage
  8                  HAVING SUM(blocks)> 10000 -- 80MB
  9                  GROUP BY username, sqladdr, sqlhash, sql_id, tablespace, session_addr
 10          ) "VSU",
 11          v$sqltext vst,
 12          v$session vs,
 13          v$process vp
 14  WHERE vsu.sql_id = vst.sql_id
 15          AND vsu.sqladdr = vst.address
 16          AND vsu.sqlhash = vst.hash_value
 17          AND vsu.session_addr = vs.saddr
 18          AND vs.paddr = vp.addr
 19          AND vst.piece = 0;
 
no rows selected
 
SYS@LAB1024> -- note the filter to only get sorts using more than 10000 blocks of temp space
SYS@LAB1024> -- (that's why no records were returned)
SYS@LAB1024> -- go to another session and start a big sort, then check again
 
SYS@LAB1024> /
 
TIME_STAM USERNAME        SQL_ID        TABLESPACE        USAGE_MB SQL_TEXT                                                         SPID
--------- --------------- ------------- --------------- ---------- ---------------------------------------------------------------- ------------
19-JAN-10 SYS             9babjv8yq8ru3 TEMP                   145 BEGIN DBMS_OUTPUT.GET_LINES(:LINES, :NUMLINES); END;             31514
 
SYS@LAB1024> -- so now we have a big one
SYS@LAB1024> -- but obviously the get_lines procedure is not causing the big sort
SYS@LAB1024> -- hmmm, what's actually active right now?
SYS@LAB1024> @as
 
no rows selected
 
SYS@LAB1024> l
  1  select sid, substr(program,1,19) prog, address, hash_value, b.sql_id, sql_child_number child, plan_hash_value, executions execs,
  2  (elapsed_time/decode(nvl(executions,0),0,1,executions))/1000000 avg_etime,
  3  sql_text
  4  from v$session a, v$sqlarea b
  5  where status = 'ACTIVE'
  6  and username is not null
  7  and a.sql_id = b.sql_id
  8* and audsid != SYS_CONTEXT('userenv','sessionid')
SYS@LAB1024> del
SYS@LAB1024> /
 
 SID PROG       ADDRESS  HASH_VALUE SQL_ID         CHILD PLAN_HASH_VALUE      EXECS   AVG_ETIME SQL_TEXT
---- ---------- -------- ---------- ------------- ------ --------------- ---------- ----------- -----------------------------------------
 410 sqlplus@ho 74AAF3D0 1572343862 1mkjj1tfvh41q      0      1304592819          5       67.42 SELECT COL1,COL2 FROM KSO.SKEW ORDER BY C
 433 sqlplus@ho 74BB6C20 1310159811 bn8b1wp71fwy3      0       187045271          4         .01 select sid, substr(program,1,19) prog, ad
 
SYS@LAB1024> -- yep, SID 410 is running the query I started in the other session
SYS@LAB1024> -- let' check the prev_sql_id as indicated by the Metalink note
SYS@LAB1024>
SYS@LAB1024> select sid, sql_id, prev_sql_id from v$session where sid=410;
 
 SID SQL_ID        PREV_SQL_ID
---- ------------- -------------
 410 1mkjj1tfvh41q 9babjv8yq8ru3
 
SYS@LAB1024> -- aha, it is prev_sql_id that is exposed in v$sort_usage
SYS@LAB1024> -- just to prove it let's look at the fixed view definition
SYS@LAB1024> get fixed_view_def
  1  select * from v$fixed_view_definition
  2* where view_name like upper('&view_name')
SYS@LAB1024> /
Enter value for view_name: GV$SORT_USAGE
 
VIEW_NAME
------------------------------
VIEW_DEFINITION
-----------------------------------------------------------------------------------------------------------------------------------------------------------
GV$SORT_USAGE
select x$ktsso.inst_id, username, username, ktssoses, ktssosno, prev_sql_addr, prev_hash_value, prev_sql_id, ktssotsn, decode(ktssocnt, 0, 'PERMANENT', 1,
'TEMPORARY'), decode(ktssosegt, 1, 'SORT', 2, 'HASH', 3, 'DATA', 4, 'INDEX', 5, 'LOB_DATA', 6, 'LOB_INDEX' , 'UNDEFINED'), ktssofno, ktssobno, ktssoexts, k
tssoblks, ktssorfno from x$ktsso, v$session where ktssoses = v$session.saddr and ktssosno = v$session.serial#
 
SYS@LAB1024> -- notice the prev_sql_id ...
SYS@LAB1024>
SYS@LAB1024> -- so now let's look at a fixed version (only modified slightly)
SYS@LAB1024> -- since we're already joining to v$session, we can just get the sql_id from there
SYS@LAB1024> -- I rearranged the columns a bit as well 
SYS@LAB1024> -- and joined to dba_tablespaces to allow for various block sizes
SYS@LAB1024> 
SYS@LAB1024> @temp_usage
 
no rows selected
 
SYS@LAB1024> l
  1  SELECT sysdate "TIME_STAMP", vsu.username, vs.sid, vp.spid, vs.sql_id, vst.sql_text, vsu.tablespace,
  2         sum_blocks*dt.block_size/1024/1024 usage_mb
  3     FROM
  4     (
  5             SELECT username, sqladdr, sqlhash, sql_id, tablespace, session_addr,
  6  -- sum(blocks)*8192/1024/1024 "USAGE_MB",
  7                  sum(blocks) sum_blocks
  8             FROM v$sort_usage
  9             HAVING SUM(blocks)> 1000
 10             GROUP BY username, sqladdr, sqlhash, sql_id, tablespace, session_addr
 11     ) "VSU",
 12     v$sqltext vst,
 13     v$session vs,
 14     v$process vp,
 15     dba_tablespaces dt
 16  WHERE vs.sql_id = vst.sql_id
 17  -- AND vsu.sqladdr = vst.address
 18  -- AND vsu.sqlhash = vst.hash_value
 19     AND vsu.session_addr = vs.saddr
 20     AND vs.paddr = vp.addr
 21     AND vst.piece = 0
 22     AND dt.tablespace_name = vsu.tablespace
 23* order by usage_mb
 
SYS@LAB1024> -- start the sort again, and recheck
SYS@LAB1024> /
 
TIME_STAM USERNAME         SID SPID         SQL_ID        SQL_TEXT                                           TABLESPACE        USAGE_MB
--------- --------------- ---- ------------ ------------- -------------------------------------------------- --------------- ----------
19-JAN-10 SYS              410 31514        1mkjj1tfvh41q SELECT COL1,COL2 FROM KSO.SKEW ORDER BY COL3, COL2 TEMP                   191
 
SYS@LAB1024> -- that looks better, that's the statement that is causing the big sort and temp usage

As pointed out in the Metalink note, it’s possible that the current sql_id may not have the correct value in some situations. One specific case that was mentioned was when a statement has completed and the session has issued another statement, but the temp space has not yet been cleaned up. It seems to work pretty well in most situations though.

One other note, it appears that this issue (bug) has not been addressed as of 11.2.0.1.

Autotuned DB_FILE_MULTIBLOCK_READ_COUNT

osborne — Wed, 13 Jan 2010 02:03:33 +0000

This topic keeps coming up so I decided to write a little blurb on it. Mainly so I could stop repeating myself (I think I’ve written the same email about 10 times now). So here goes:

First the short version:

If you are on 10gR2 or later you should probably not set DB_FILE_MULTIBLOCK_READ_COUNT.

Now the long winded discussion:

A fairly significant change to the Oracle optimizer was rolled out in 10gR2 with regards to the parameter DB_FILE_MULTIBLOCK_READ_COUNT (from here on out I’ll just refer to as DBFMBRC as it’s too long to type). Prior to 10gR2, this parameter was used for multiple purposes - namely:

Representing the avg number of blocks that a multi-block read would return (used in the costing estimate when parsing)
The number of blocks to attempt to read when a multi-block read is executed

Unfortunately, these two uses often worked against each other. Increasing the DBFMBRC in order to improve the throughput of multi-block reads generated by operations such as full table scans, would have the side effect of causing the optimizer to favor access paths that resulted in multi-block reads. I think that was probably the main driver for the introduction of the OPTIMIZER_INDEX_COST_ADJ and OPTIMIZER_INDEX_CACHING parameters - which allowed the optimizer’s preference for full table scans to be dialed down, but that’s a topic for another post.

The costing value and the execution value need not necessarily be related (and in 10gR2 they aren’t unless you make it so). For example, the estimate for costing purposes could be set to a relatively low value (so that the optimizer would not unduly favor multi-block read type operations) while the value used when actually requesting a multi-block i/o could be set relatively high (to maximize throughput in cases where a multi-block read was actually performed). 10gR2 accomplishes this by adding a couple of so called hidden parameters (you know, the one’s that start with an underscore). The parameters are:

_db_file_optimizer_read_count - this is the one that’s used for costing
_db_file_exec_read_count - this is the one that’s used when an i/o request is issued

If you leave DB_FILE_MULTIBLOCK_READ_COUNT set to its default value, these two parameters will be set to two different values. For example, on my 10.2.0.4 database using 8K blocks, the values are 8 and 128 respectively. So the optimizer should do it’s costing with a value of 8 but attempt to read 128 blocks, if it does indeed choose a full table scan. Let’s take a look:

^?View Code NONE

 
> !sql
sqlplus "/ as sysdba"
 
SQL*Plus: Release 10.2.0.4.0 - Production on Wed Jan 13 17:21:55 2010
 
Copyright (c) 1982, 2007, Oracle.  All Rights Reserved.
 
 
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
 
SYS@LAB1024> select     pname, pval1
  2  from       sys.aux_stats$
  3  /
 
PNAME                               PVAL1
------------------------------ ----------
STATUS
DSTART
DSTOP
FLAGS                                   1
CPUSPEEDNW                     526.069569
IOSEEKTIM                              10
IOTFRSPEED                           4096
SREADTIM
MREADTIM
CPUSPEED
MBRC
MAXTHR
SLAVETHR
 
13 rows selected.
 
SYS@LAB1024> -- default values
SYS@LAB1024> 
SYS@LAB1024> select name, value, isdefault, ismodified, isset
  2  from
  3  (
  4  select flag,name,value,isdefault,ismodified,
  5  case when isdefault||ismodified = 'TRUEFALSE' then 'FALSE' else 'TRUE' end isset
  6  from
  7     (
  8         select
  9              decode(substr(i.ksppinm,1,1),'_',2,1) flag
 10              , i.ksppinm name
 11              , sv.ksppstvl value
 12              , sv.ksppstdf  isdefault
 13  --            , decode(bitand(sv.ksppstvf,7),1,'MODIFIED',4,'SYSTEM_MOD','FALSE') ismodified
 14              , decode(bitand(sv.ksppstvf,7),1,'TRUE',4,'TRUE','FALSE') ismodified
 15           from x$ksppi  i
 16              , x$ksppsv sv
 17          where i.indx = sv.indx
 18     )
 19  )
 20  where name like nvl('%¶meter%',name)
 21  and upper(isset) like upper(nvl('%&isset%',isset))
 22  and flag not in (decode('&show_hidden','Y',3,2))
 23  order by flag,replace(name,'_','')
 24  /
Enter value for parameter: read_count
Enter value for isset: 
Enter value for show_hidden: Y
 
NAME                                               VALUE                                                                  ISDEFAUL ISMODIFIED ISSET
-------------------------------------------------- ---------------------------------------------------------------------- -------- ---------- ----------
db_file_multiblock_read_count                      128                                                                    TRUE     FALSE      FALSE
_db_file_exec_read_count                           128                                                                    TRUE     FALSE      FALSE
_db_file_noncontig_mblock_read_count               11                                                                     TRUE     FALSE      FALSE
_db_file_optimizer_read_count                      8                                                                      TRUE     FALSE      FALSE
_sort_multiblock_read_count                        2                                                                      TRUE     FALSE      FALSE
 
SYS@LAB1024> -- default values again
SYS@LAB1024> 
SYS@LAB1024> -- let's turn on 10046 trace
SYS@LAB1024> 
SYS@LAB1024> set echo on 
SYS@LAB1024> @find_trace
SYS@LAB1024> col tracefile_name for a120
SYS@LAB1024> SELECT rtrim(k.value,'/')||'/'||LOWER(d.instance_name)||'_ora_'||p.spid
  2  ||DECODE(p.value,'','','_'||p.value)||'.trc' tracefile_name
  3  FROM v$parameter k, v$parameter p, v$instance d,
  4       sys.v_$session s, sys.v_$process p,
  5       (SELECT sid FROM v$mystat WHERE rownum=1) m
  6  WHERE p.name = 'tracefile_identifier'
  7    AND k.name = 'user_dump_dest'
  8    AND s.paddr = p.addr
  9    AND s.sid = m.sid
 10  /
 
TRACEFILE_NAME
------------------------------------------------------------------------------------------------------------------------
/u01/app/admin/LAB1024/udump/lab1024_ora_24004.trc
 
 
SYS@LAB1024> alter session set events '10046 trace name context forever, level 8';
 
Session altered.
 
SYS@LAB1024> set echo off
SYS@LAB1024> select avg(pk_col) from kso.skew a where col1 > 0;
select avg(pk_col) from kso.skew a where col1 > 0
                            *
ERROR at line 1:
ORA-01013: user requested cancel of current operation
 
 
SYS@LAB1024> -- bombed out with ctl-C 
SYS@LAB1024> -- now find the sql_id and verify the plan
SYS@LAB1024>
SYS@LAB1024> @find_sql
Enter value for sql_text: select avg(pk_col) from kso.skew a where col1 > 0
Enter value for sql_id: 
 
SQL_ID         CHILD  PLAN_HASH      EXECS     AVG_ETIME      AVG_LIO SQL_TEXT
------------- ------ ---------- ---------- ------------- ------------ ------------------------------------------------------------
05cq2hb1r37tr      0  568322376          1        183.30       35,082 select avg(pk_col) from kso.skew a where col1 > 0
 
 
SYS@LAB1024> @dplan
Enter value for sql_id: 05cq2hb1r37tr
Enter value for child_no: 0
 
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  05cq2hb1r37tr, child number 1
-------------------------------------
select avg(pk_col) from kso.skew a where col1 > 0
 
Plan hash value: 568322376
 
---------------------------------------------------------------------------
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |       |       | 45254 (100)|          |
|   1 |  SORT AGGREGATE    |      |     1 |    11 |            |          |
|*  2 |   TABLE ACCESS FULL| SKEW |    32M|   335M| 45254   (3)| 00:09:04 |
---------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   2 - filter("COL1">0)
 
SYS@LAB1024> -- we can verify that it read 128 blocks at a time by looking at the trace file
SYS@LAB1024>
SYS@LAB1024> !cat /u01/app/admin/LAB1024/udump/lab1024_ora_24004.trc
 
...
 
PARSING IN CURSOR #6 len=49 dep=0 uid=0 oct=3 lid=0 tim=1233806813773567 hv=3279003447 ad='7e81b9b4'
select avg(pk_col) from kso.skew a where col1 > 0
END OF STMT
PARSE #6:c=213967,e=209976,p=27,cr=287,cu=0,mis=1,r=0,dep=0,og=2,tim=1233806813773550
EXEC #6:c=0,e=77,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=2,tim=1233806813773842
WAIT #6: nam='SQL*Net message to client' ela= 10 driver id=1650815232 #bytes=1 p3=0 obj#=472 tim=1233806813773941
WAIT #6: nam='db file sequential read' ela= 81 file#=4 block#=181290 blocks=1 obj#=54141 tim=1233806813791873
WAIT #6: nam='db file scattered read' ela= 5218 file#=4 block#=181291 blocks=128 obj#=54141 tim=1233806813800264
WAIT #6: nam='db file scattered read' ela= 3239 file#=4 block#=181419 blocks=128 obj#=54141 tim=1233806814404594

Note 1: Very nice! Maximum i/o without making the optimizer think that full table scans are better than index access paths.

Note 2: Setting the parameter to the default value is not the same as resetting it. If you were to change the value to 16 for example and then set it back to 8 (which was the value it defaulted to) you would see the costing and the execution done with 8. That is, the _db_file_optimizer_read_count and the _db_file_exec_read_count would both be set to 8. You have to unset the DBFMBRC parameter to get it to go back to automatic split of 8 and 128. See my previous post on that here: Resetting Oracle Parameters Here’s another quick demonstration:

^?View Code NONE

SYS@LAB1024> @parms
Enter value for parameter: read_count
Enter value for isset: 
Enter value for show_hidden: Y
 
NAME                                               VALUE                                                                  ISDEFAUL ISMODIFIED ISSET
-------------------------------------------------- ---------------------------------------------------------------------- -------- ---------- ----------
db_file_multiblock_read_count                      128                                                                    TRUE     FALSE      FALSE
_db_file_exec_read_count                           128                                                                    TRUE     FALSE      FALSE
_db_file_noncontig_mblock_read_count               11                                                                     TRUE     FALSE      FALSE
_db_file_optimizer_read_count                      8                                                                      TRUE     FALSE      FALSE
_sort_multiblock_read_count                        2                                                                      TRUE     FALSE      FALSE
 
SYS@LAB1024> alter system set db_file_multiblock_read_count=16;
 
System altered.
 
SYS@LAB1024> @parms
Enter value for parameter: read_count
Enter value for isset: 
Enter value for show_hidden: Y
 
NAME                                               VALUE                                                                  ISDEFAUL ISMODIFIED ISSET
-------------------------------------------------- ---------------------------------------------------------------------- -------- ---------- ----------
db_file_multiblock_read_count                      16                                                                     TRUE     TRUE       TRUE
_db_file_exec_read_count                           16                                                                     TRUE     FALSE      FALSE
_db_file_noncontig_mblock_read_count               11                                                                     TRUE     FALSE      FALSE
_db_file_optimizer_read_count                      16                                                                     TRUE     FALSE      FALSE
_sort_multiblock_read_count                        2                                                                      TRUE     FALSE      FALSE
 
SYS@LAB1024> alter system set db_file_multiblock_read_count=128;
 
System altered.
 
SYS@LAB1024> @parms
Enter value for parameter: read_count
Enter value for isset: 
Enter value for show_hidden: Y
 
NAME                                               VALUE                                                                  ISDEFAUL ISMODIFIED ISSET
-------------------------------------------------- ---------------------------------------------------------------------- -------- ---------- ----------
db_file_multiblock_read_count                      128                                                                    TRUE     TRUE       TRUE
_db_file_exec_read_count                           128                                                                    TRUE     FALSE      FALSE
_db_file_noncontig_mblock_read_count               11                                                                     TRUE     FALSE      FALSE
_db_file_optimizer_read_count                      128                                                                    TRUE     FALSE      FALSE
_sort_multiblock_read_count                        2                                                                      TRUE     FALSE      FALSE
 
SYS@LAB1024> alter system set db_file_multiblock_read_count=0;
 
System altered.
 
SYS@LAB1024> @parms
Enter value for parameter: read_count
Enter value for isset: 
Enter value for show_hidden: Y
 
NAME                                               VALUE                                                                  ISDEFAUL ISMODIFIED ISSET
-------------------------------------------------- ---------------------------------------------------------------------- -------- ---------- ----------
db_file_multiblock_read_count                      128                                                                    TRUE     FALSE      FALSE
_db_file_exec_read_count                           128                                                                    TRUE     FALSE      FALSE
_db_file_noncontig_mblock_read_count               11                                                                     TRUE     FALSE      FALSE
_db_file_optimizer_read_count                      128                                                                    TRUE     FALSE      FALSE
_sort_multiblock_read_count                        2                                                                      TRUE     FALSE      FALSE

So once you set the value of DBFMBRC, the exec and read parms will be the same.

Note 3: The _db_file_noncontig_mblock_read_count has to do with index block pre-fetching. There is an excellent description on this oracle-l post by Tanel Poder

Note 4: Everything I’ve said so far assumes that System Statistics have not been set or gathered. If System Stats are set, then a completely different set of rules kick in - basically DBFMBRC is ignored.

And speaking of System Stats, I must say that I think the default no-workload System Stats work pretty well. In most cases, I prefer not to gather or set System Statistics, for a couple of reasons:

In the first place, I generally don’t believe in fixing things that aren’t broken. You could argue that System Statistics are not meant to “fix” anything, but are merely designed to give the optimizer more specific information about the particular system. And this is true, but the standard method of gathering Workload System Statistics often results in less than accurate results and at best they only represent an average over some period of time which may or may not be representative of what happens when the system is really under stress. And I guess we could have a few sets of values and change them regularly (like one set for batch processing and one for day time on-line processing), but that seems like a lot of complexity for little potential benefit. So unless there really is a specific problem that needs to be solved, I probably wouldn’t do that (in fact I have not ever done that).

Second, I would venture to guess that the majority of databases out there have not gathered or set System Statistics. I could be way off on that assumption, but most of the systems that I’ve looked at over the last couple of years have not had anything other than the default NOWORKLOAD values. Staying in the middle of the pack means we’re less likely to run into weird behavior that someone else hasn’t already run across.

Third, the interaction between the various flavors of System Statistics and the DBFMBRC are complicated. Randolf Geist has an excellent series of posts which discusses the interaction between System Stats and DFMBRC here: RG on System Stats
But it takes 3 fairly long and detailed posts to cover the topic. I am not going to discuss it here since he has already done a very thorough job.

Fourth, System Statistics is (are) a really big knob. What I mean by that is that a small change can have a profound effect. Since they control the costing of every single statement that runs through your system, changes to them could potentially affect every statement. So from a stability stand point, this is not something that you would want to change without careful thought and certainly not something that you would want to change on a regular basis.

Now for a couple of references:

Jonathan Lewis has several posts on his blog about System Stats. Here are links to a couple:

JL on System Stats
JL on System Stats 2

Be sure and read through the comments as there is some excellent information there including how the default values for the read and exec parameters are arrived at.

Finally, Tom Kyte has a good discussion on the subject here: TK on DBFMBRC

Hotsos Symposium 2010

osborne — Mon, 04 Jan 2010 15:02:45 +0000

The best Oracle conference in the universe.

Well this will make 5 years in row that I’ve attended the Hotsos Symposium (this will be my third time as a speaker). As I’ve said before, it’s been far and away the best conference or training event that I’ve ever participated in. Add to that the extremely high quality of the people in attendance and you get an awesome event. There are always a fair number of people in the audience that could be delivering the presentations. One of the best things about it is the impromptu conversations that invariably pop up after (or sometimes during) a presentation.

The 2010 Symposium will be held in Dallas (as usual), March 7-11. Tom Kyte will be giving the key note address. Tom is well known, and rightly so, as he is always thought provoking and entertaining. Tanel Põder will be delivering the optional training day. If you haven’t heard of him yet you should do yourself a favor and check out his blog. There are only a handful of guys in the world that understand Oracle internals as well as Tanel.

I must say I am extremely honored to be able to present again at this years symposium. Here’s a link to the speakers page for this years event which has links to the abstracts for their presentations. And here’s a link to the main Symposium page where you can find info on how to sign up.

Here’s the list of speakers (in case it’s too much trouble to click the link above):

Alex Gorbachev - Battle Against Any Guess & Run-Time Load Balancing in Oracle RAC
Alex Haralampiev - When a Good Design Goes Bad
Andrew Zitelli - Oracle 11g “Partitioning by Reference” - The Advantages and Annoyances
Bryn Llewellyn - Edition-Based Redefinition: the Key to Online Application Upgrade
Cary Millsap - Lessons Learned - Version 2010.03
Christian Antognini - Diagnosing Parallel Executions Performance
Dan Norris - Consolidation Strategies for Oracle Database Machine
Dave Abercrombie - End-to-End Metrics for Troubleshooting and Monitoring
Doug Burns - Odyssey Two: Parallel Query in 2010
Henry Poras - Diminishing Resource Utilization and Saturation Limits Using AWR History and Queueing Theory
Kerry Osborne - Controlling Execution Plans (without Touching the Code)
Kevin Closson - TBA
Kevin Williams - How We Dealt with the Chronic Problem of Too Much Data on a Large OLTP System
Kyle Hailey - Modern Approaches to SQL Tuning
Marco Gralike - The Ultimate Performance Challenge: How to Make XML Perform.?!
Mark Bobak - A Closer Look at Parsing: Possible Application Optimizations
Monty Orme - TBA
Neil Gunther & Peter Stalder - TBA
Ric Van Dyke - TBA
Richard Foote - Oracle Indexing Myths & Oracle Indexing Tricks and Traps
Richard McDougall - Performance and Sizing of Oracle on VMware
Riyaj Shamsudeen - A Close Encounter with Real World (and Odd) Performance Issues & Why Does Optimizer Hate My SQL?
Stephan Haisley - Streams, Xstreams and Golden Gate
Tanel Põder - TBA
Tom Kyte - All About Metadata; Why Telling the Database About Your Schema Matters
& Efficient PL/SQL — Why and How to Use PL/SQL to Its Greatest Effect
Vlado Barun & Edwin Putkonen - Deploying Database Changes: Performance Matters
Wolfgang Breitling - Anatomy of a SQL Tuning Session & Seeding Statistics

Just as a side note, I got an opportunity to speak at last years Hotsos Symposium. One of the functions at the Symposium is a social gathering which provides a great opportunity to talk to a bunch of really smart guys in a less formal setting. Unfortunately, a few of the participants over indulge at the party. Fortunately, many of them have a room at the host hotel (so no driving). Unfortunately, I ended up speaking at the first session the morning after the party. The audience looked a little like this:

Actually it wasn’t that bad, but I am looking forward to a better time slot this time around!

Hope to see you there!

My Dad’s in the Hall of Fame

osborne — Thu, 31 Dec 2009 15:49:23 +0000

My dad (Roddy Osborne) got inducted into the Texas A&M Sports Hall of Fame a couple of months ago. My dad played for Bear Bryant back in the late 50’s. I knew he was a quarterback, but I didn’t realize that he played fullback his sophmore year. Turns out he was also the punter and the punt return guy (who knew). Oh yeah, and did I mention, in Bear Bryant’s system, the quarterback played middle linebacker too! He got drafted by the Cleveland Browns and played a couple of seasons there. (I think there was a guy by the name of Jim Brown playing there then).

Anyway, here’s the video that the committee put together. They played it before his acceptance speach at the banquet. By the way, that’s John David Crow (Heisman Trophy winner in 1957) narrating. It has some good footage of my dad playing (I didn’t know he could pass!).

My dad didn’t talk much about football when we were growing up. He actually didn’t want my brother and me to play because it was so hard on his body. He had huge scars on his knees and one on his shoulder that went from the front arm pit all the way around to the back arm pit. That scar was like an inch wide with cross hatches every couple of inches. He was so beat up that when his Air Force company was shipped out to Vietnam, they wouldn’t allow him to go. Anyway, there are only a couple of stories that I remember him telling:

The first one occurred during a game between Texas A&M and Arkansas in 1957. The story has been repeated many times over the years (often incorrectly). I once heard a preacher tell the story in a sermon and he had it completely messed up. I had to straighten him out after the service. Anyway, here’s how I remember it:

The Aggies were ahead by one point with a couple of minutes to go in the game. They were driving and close to scoring. Back in those days, the coach didn’t call the plays. The quarterback was responsible for calling the plays. So coach Bryant sends in a substitute with a message to not throw the ball, just to run out the clock. My dad calls a roll out, expecting to burn some clock with a long, slow developing play, but not intending to throw the ball. So he rolls out and lo and behold, a receiver (I think he said it was Gene Stallings) is wide open in the end zone. He told me that the guy was so open he just couldn’t help himself and so he throws the ball. Well one of the Arkansas defensive backs (Donny Horton) is a sprinter on the track team and he breaks on the ball along the sidelines and intercepts it. And there is nobody between him and the end zone 90+ yards away. Well somehow, my dad catches this guy from behind, and my dad is reportedly one of the slowest guys on the Aggies team. That tackle saves their undefeated season (they were 7-0 at the time and #1 in the nation). Later Bear Bryant was asked about the play and he says, “The difference was that Horton was running for a touchdown. Osborne was running for his life.”

The second story he told that stuck with me was about the last day he played football.

After playing a couple of years in Cleveland, my dad was drafted and entered the Air Force. He was stationed in Bitburg Germany. Apparently they had some sort of semi-pro league there and my dad was a player coach. The story I remember went like this (I’ll tell it from his perspective):

Did I ever tell you about the last time I played football? Well I was a player coach in Bitburg, so I suited up, but never played because I was so out of shape. Well our first string quarterback got hurt and so I put in our second string guy. And I told him, what ever you do, don’t get yourself hurt. Because we don’t have another quarterback except me. And sure enough, on the second play, the guy takes off on a run and gets clobbered and I have to go in. Well we’re backed up close to our end zone and I call a pass play and I roll out to the right and it’s like the seas parting. There is no one between me and the end zone.

So I take off running…

And I’m running, and running, and running…

And I look over and the referee is keeping up with me…

Running backwards…

And on about the 10 yard line, a big old fat lineman catches me from behind…

And I roll over on the sidelines…

And I throw up…

And that’s the last time I played football.

Several other guys got inducted at the same time. Greg Hill and Quintin Coryatt were the two that made the biggest impression on me at the banquet. Greg Hill was a great running back in early 90’s that was drafted in the first round by Kansas City. Quintin Coryatt was a linebacker in the early 90’s. As part of the Wrecking Crew, he was probably most well known for “The Hit” which was aired repeatedly after a nationally televised game against TCU. He was drafted with the number 2 overall pick by the Indianapolis Colts. Here’s a video of “The Hit”.

Tracking Parameter Changes

osborne — Fri, 11 Dec 2009 16:40:25 +0000

I was in a meeting yesterday and a guy asked if there was a way to track changes to database parameters (sometimes called init.ora parameters by us old guys). I thought for a second and said I didn’t think there was any sort of built in mechanism for doing that, no history table that I was aware of. Then one of the other guys in the meeting said, “oh yeah, we do that by looking at the AWR table that has the list of non-default parameter settings” (that table is WRH$_PARAMETER by the way).

Hmmm, interesting approach. AWR runs every hour by default. Not a bad idea. They had a script that prompted for a parameter name and did a dump of all entries for the specified parameter. So you could easily see where a change had occurred.

Looks like this:

^?View Code NONE

SYS@LAB1024> @parm_hist 
Enter value for pname: star_transformation_enabled
 
   SNAP_ID TIME            PARAMETER_NAME                                     VALUE
---------- --------------- -------------------------------------------------- --------------------
      2232 25-SEP-09 00    star_transformation_enabled                        FALSE
      2233 25-SEP-09 01    star_transformation_enabled                        FALSE
      2234 25-SEP-09 02    star_transformation_enabled                        FALSE
      2235 25-SEP-09 03    star_transformation_enabled                        FALSE
      2376 01-OCT-09 00    star_transformation_enabled                        FALSE
      2377 01-OCT-09 01    star_transformation_enabled                        FALSE
      2378 01-OCT-09 02    star_transformation_enabled                        FALSE
      2379 01-OCT-09 03    star_transformation_enabled                        FALSE
      2380 01-OCT-09 04    star_transformation_enabled                        FALSE
      2381 01-OCT-09 05    star_transformation_enabled                        FALSE
      2382 01-OCT-09 06    star_transformation_enabled                        FALSE
      2383 01-OCT-09 07    star_transformation_enabled                        FALSE
      2384 01-OCT-09 08    star_transformation_enabled                        FALSE
      2385 01-OCT-09 09    star_transformation_enabled                        FALSE
      2386 01-OCT-09 10    star_transformation_enabled                        FALSE
      2387 01-OCT-09 11    star_transformation_enabled                        FALSE
      3900 02-DEC-09 23    star_transformation_enabled                        TRUE
      3901 03-DEC-09 00    star_transformation_enabled                        TRUE
      3902 03-DEC-09 01    star_transformation_enabled                        TRUE
      3903 03-DEC-09 02    star_transformation_enabled                        TRUE
      3904 03-DEC-09 03    star_transformation_enabled                        TRUE
      3905 03-DEC-09 04    star_transformation_enabled                        TRUE
      3906 03-DEC-09 05    star_transformation_enabled                        TRUE
      3907 03-DEC-09 06    star_transformation_enabled                        TRUE
...

Of course my first thought was “that’s nice, but most systems only have a few weeks of history in AWR”. I’ll come back to that issue in a minute, but for now let’s go on to the part where my brain started working on ways to make use of this idea and to maybe improve on it a little bit.

To begin with, I didn’t want to see a record for every snapshot if nothing had changed. I would prefer to just see a single record with both the old and new value when there was actually a change. Easy enough to do with the an analytic query using the lag function. I also thought I’d like to be able to wild card the parameter - no problem there. Then I decided I wanted it to be RAC aware and let me specify a single instance (since some of the parameters have different values depending on the instance). And finally, I found the calculated hidden parameters to be annoying (the ones that start with 2 underscores like “__shared_pool_size”). Several of them get reset on a regular basis, and I am not usually all that interested in those. So I added a switch to turn them off (or not). I called the script parm_mods.sql.

Here’s a quick example:

^?View Code NONE

 
SYS@LAB1024> @parm_mods
Enter value for parameter_name: 
Enter value for instance_number: 
Enter value for show_calculated: 
 
  INSTANCE    SNAP_ID TIME            PARAMETER_NAME                      OLD_VALUE            NEW_VALUE
---------- ---------- --------------- ----------------------------------- -------------------- --------------------
         1       2376 01-OCT-09 00:00 db_recovery_file_dest_size          26843545600          42949672960
                 3900 02-DEC-09 23:00 db_file_multiblock_read_count       16                   128
                 3900 02-DEC-09 23:00 parallel_execution_message_size     2148                 8192
                 3900 02-DEC-09 23:00 hash_area_size                      131072               200000000
                 3900 02-DEC-09 23:00 large_pool_size                     0                    536870912
                 3900 02-DEC-09 23:00 workarea_size_policy                AUTO                 MANUAL
                 3900 02-DEC-09 23:00 shared_pool_reserved_size           12582912             24326963
                 3900 02-DEC-09 23:00 shared_pool_size                    0                    419430400
                 3900 02-DEC-09 23:00 sort_area_size                      65536                100000000
                 3900 02-DEC-09 23:00 star_transformation_enabled         FALSE                TRUE
                 4085 10-DEC-09 15:02 _spin_count                         2001                 2002
 
 
 
 
11 rows selected.
 
SYS@LAB1024> /
Enter value for parameter_name: 
Enter value for instance_number: 
Enter value for show_calculated: Y
 
  INSTANCE    SNAP_ID TIME            PARAMETER_NAME                      OLD_VALUE            NEW_VALUE
---------- ---------- --------------- ----------------------------------- -------------------- --------------------
         1       2376 01-OCT-09 00:00 __shared_pool_size                  251658240            285212672
                 2376 01-OCT-09 00:00 db_recovery_file_dest_size          26843545600          42949672960
                 2376 01-OCT-09 00:00 __db_cache_size                     1275068416           1241513984
                 3900 02-DEC-09 23:00 db_file_multiblock_read_count       16                   128
                 3900 02-DEC-09 23:00 __shared_pool_size                  285212672            486539264
                 3900 02-DEC-09 23:00 __large_pool_size                   16777216             536870912
                 3900 02-DEC-09 23:00 workarea_size_policy                AUTO                 MANUAL
                 3900 02-DEC-09 23:00 __db_cache_size                     1241513984           520093696
                 3900 02-DEC-09 23:00 hash_area_size                      131072               200000000
                 3900 02-DEC-09 23:00 large_pool_size                     0                    536870912
                 3900 02-DEC-09 23:00 parallel_execution_message_size     2148                 8192
                 3900 02-DEC-09 23:00 shared_pool_reserved_size           12582912             24326963
                 3900 02-DEC-09 23:00 shared_pool_size                    0                    419430400
                 3900 02-DEC-09 23:00 sort_area_size                      65536                100000000
                 3900 02-DEC-09 23:00 star_transformation_enabled         FALSE                TRUE
                 4085 10-DEC-09 15:02 _spin_count                         2001                 2002
 
 
 
 
16 rows selected.

So back to the issue of AWR retention…

I think this would be a very good way of keeping a record of parameter changes if the retention was long enough. The default is to keep only 7 days of history, although many systems keep a month or two. Turns out, the guys that are doing this keep 7 years of AWR history (actually they only have 2 years so far, but retention is set to 7 years). That made me wonder how much space 7 years would take and if performance would suffer on any of the standard AWR stuff. I checked a couple of systems and came up with an average of around 1 Meg per snapshot. So that’s about 168 Megs per week, 720 Megs per month, around 9 Gigs per year. They reported a much lower number, only about 4G for 2 years. I am not sure I believe that since they have a 3 node RAC environment (so 3X the data). The smallest number I found on any of the systems I checked was about 0.5 Megs per snap on a database that is virtually idle. The SQL Plans and the Active Session History take the most space. So very complex statements would tend to increase the size and obviously more activity would increase the size. See here:

^?View Code NONE

 
SQL> select count(*) from dba_hist_snapshot;
 
  COUNT(*)
----------
      1841
 
SQL> @table_size
Enter value for owner: 
Enter value for table_name: WRH%
 
OWNER                SEGMENT_NAME                   TOTALSIZE_MEGS TABLESPACE_NAME
-------------------- ------------------------------ -------------- ------------------------------
SYS                  WRH$_FILESTATXS_BL                         .2 SYSAUX
SYS                  WRH$_OPTIMIZER_ENV                         .2 SYSAUX
SYS                  WRH$_BUFFER_POOL_STATISTICS                .4 SYSAUX
SYS                  WRH$_SGASTAT_BL                            .3 SYSAUX
SYS                  WRH$_CURRENT_BLOCK_SERVER                  .3 SYSAUX
SYS                  WRH$_TABLESPACE_STAT_BL                    .2 SYSAUX
SYS                  WRH$_SERVICE_NAME                          .2 SYSAUX
SYS                  WRH$_WAITSTAT_BL                           .2 SYSAUX
SYS                  WRH$_SYSTEM_EVENT                        29.1 SYSAUX
SYS                  WRH$_LATCH_MISSES_SUMMARY                43.1 SYSAUX
SYS                  WRH$_PARAMETER                           34.1 SYSAUX
SYS                  WRH$_SERVICE_WAIT_CLASS                   4.6 SYSAUX
SYS                  WRH$_TABLESPACE_STAT                      3.4 SYSAUX
SYS                  WRH$_SQL_BIND_METADATA_PK                 5.0 SYSAUX
SYS                  WRH$_WAITSTAT_BL_PK                        .1 SYSAUX
SYS                  WRH$_SGA_PK                                .6 SYSAUX
SYS                  WRH$_SHARED_POOL_ADVICE_PK                2.0 SYSAUX
SYS                  WRH$_SQL_WORKAREA_HIST_PK                 3.0 SYSAUX
SYS                  WRH$_JAVA_POOL_ADVICE_PK                   .8 SYSAUX
SYS                  WRH$_LATCH_PK                            49.1 SYSAUX
SYS                  WRH$_DB_CACHE_ADVICE_PK                   3.1 SYSAUX
SYS                  WRH$_SGASTAT_U                            4.0 SYSAUX
SYS                  WRH$_SYSSTAT_PK                          45.1 SYSAUX
SYS                  WRH$_DLM_MISC_PK                          9.3 SYSAUX
SYS                  WRH$_BUFFERED_SUBSCRIBERS                  .1 SYSAUX
SYS                  WRH$_SQL_PLAN                           240.0 SYSAUX
SYS                  WRH$_LIBRARYCACHE                         3.0 SYSAUX
SYS                  WRH$_SQL_WORKAREA_HISTOGRAM               3.0 SYSAUX
SYS                  WRH$_PARAMETER_NAME                        .1 SYSAUX
SYS                  WRH$_SESSMETRIC_HISTORY                    .1 SYSAUX
SYS                  WRH$_CR_BLOCK_SERVER                       .4 SYSAUX
SYS                  WRH$_FILESTATXS                           5.8 SYSAUX
SYS                  WRH$_OSSTAT                               2.6 SYSAUX
SYS                  WRH$_BUFFERED_SUBSCRIBERS_PK               .1 SYSAUX
SYS                  WRH$_SQL_PLAN_PK                         88.0 SYSAUX
SYS                  WRH$_SYSTEM_EVENT_BL_PK                    .6 SYSAUX
SYS                  WRH$_THREAD_PK                             .3 SYSAUX
SYS                  WRH$_CR_BLOCK_SERVER_PK                    .2 SYSAUX
SYS                  WRH$_ASH_BL_PK                            2.0 SYSAUX
SYS                  WRH$_WAITCLASSMETRIC_HIST_IND             8.0 SYSAUX
SYS                  WRH$_ACTIVE_SESSION_HISTORY_PK           94.1 SYSAUX
SYS                  WRH$_SYS_TIME_MODEL_PK                    2.7 SYSAUX
SYS                  WRH$_SQL_BIND_METADATA                    8.0 SYSAUX
SYS                  WRH$_PROCESS_MEMORY_SUMMARY                .9 SYSAUX
SYS                  WRH$_SQL_SUMMARY                           .3 SYSAUX
SYS                  WRH$_LATCH_BL                             2.0 SYSAUX
SYS                  WRH$_SEG_STAT_BL                          3.0 SYSAUX
SYS                  WRH$_METRIC_NAME                           .1 SYSAUX
SYS                  WRH$_FILEMETRIC_HISTORY                    .1 SYSAUX
SYS                  WRH$_SYSSTAT                             34.1 SYSAUX
SYS                  WRH$_ACTIVE_SESSION_HISTORY             380.1 SYSAUX
SYS                  WRH$_STREAMS_CAPTURE_PK                    .1 SYSAUX
SYS                  WRH$_COMP_IOSTAT_PK                        .1 SYSAUX
SYS                  WRH$_SQLSTAT_BL_PK                         .3 SYSAUX
SYS                  WRH$_BG_EVENT_SUMMARY_PK                  7.0 SYSAUX
SYS                  WRH$_ENQUEUE_STAT_PK                      9.0 SYSAUX
SYS                  WRH$_LATCH_CHILDREN_BL_PK                  .1 SYSAUX
SYS                  WRH$_RESOURCE_LIMIT_PK                    3.0 SYSAUX
SYS                  WRH$_LOG_PK                               3.0 SYSAUX
SYS                  WRH$_SERVICE_WAIT_CLASS_BL_PK              .1 SYSAUX
SYS                  WRH$_SYSMETRIC_HISTORY_INDEX             10.0 SYSAUX
SYS                  WRH$_SGA_TARGET_ADVICE                     .8 SYSAUX
SYS                  WRH$_STREAMS_CAPTURE                       .1 SYSAUX
SYS                  WRH$_RESOURCE_LIMIT                       3.0 SYSAUX
SYS                  WRH$_THREAD                                .3 SYSAUX
SYS                  WRH$_OSSTAT_BL                             .2 SYSAUX
SYS                  WRH$_MTTR_TARGET_ADVICE                    .1 SYSAUX
SYS                  WRH$_LATCH_CHILDREN                        .1 SYSAUX
SYS                  WRH$_SEG_STAT                            43.1 SYSAUX
SYS                  WRH$_SERVICE_STAT                        12.9 SYSAUX
SYS                  WRH$_BUFFERED_QUEUES_PK                    .1 SYSAUX
SYS                  WRH$_DATAFILE_PK                           .1 SYSAUX
SYS                  WRH$_SQLTEXT_PK                           2.0 SYSAUX
SYS                  WRH$_SQL_SUMMARY_PK                        .2 SYSAUX
SYS                  WRH$_OPTIMIZER_ENV_PK                      .1 SYSAUX
SYS                  WRH$_LATCH_NAME_PK                         .1 SYSAUX
SYS                  WRH$_DB_CACHE_ADVICE_BL_PK                 .1 SYSAUX
SYS                  WRH$_ROWCACHE_SUMMARY_BL_PK                .1 SYSAUX
SYS                  WRH$_PGA_TARGET_ADVICE_PK                 3.0 SYSAUX
SYS                  WRH$_SYSSTAT_BL_PK                         .9 SYSAUX
SYS                  WRH$_SYS_TIME_MODEL_BL_PK                  .1 SYSAUX
SYS                  WRH$_OSSTAT_BL_PK                          .1 SYSAUX
SYS                  WRH$_OSSTAT_NAME_PK                        .1 SYSAUX
SYS                  WRH$_METRIC_NAME_PK                        .1 SYSAUX
SYS                  WRH$_CURRENT_BLOCK_SERVER_PK               .1 SYSAUX
SYS                  WRH$_SGASTAT_BL_U                          .2 SYSAUX
SYS                  WRH$_SYSMETRIC_SUMMARY_INDEX             13.0 SYSAUX
SYS                  WRH$_SESSMETRIC_HISTORY_INDEX              .1 SYSAUX
SYS                  WRH$_SYSTEM_EVENT_PK                     33.1 SYSAUX
SYS                  WRH$_LATCH_PARENT_PK                       .1 SYSAUX
SYS                  WRH$_BUFFERED_QUEUES                       .1 SYSAUX
SYS                  WRH$_RULE_SET                              .3 SYSAUX
SYS                  WRH$_SESS_TIME_STATS                       .4 SYSAUX
SYS                  WRH$_COMP_IOSTAT                           .1 SYSAUX
SYS                  WRH$_SYSTEM_EVENT_BL                       .8 SYSAUX
SYS                  WRH$_BG_EVENT_SUMMARY                     6.0 SYSAUX
SYS                  WRH$_LATCH_MISSES_SUMMARY_BL              2.0 SYSAUX
SYS                  WRH$_SYS_TIME_MODEL_BL                     .2 SYSAUX
SYS                  WRH$_PARAMETER_BL                         1.0 SYSAUX
SYS                  WRH$_OSSTAT_NAME                           .1 SYSAUX
SYS                  WRH$_SQLSTAT                             66.1 SYSAUX
SYS                  WRH$_DB_CACHE_ADVICE                      4.1 SYSAUX
SYS                  WRH$_SYS_TIME_MODEL                       2.8 SYSAUX
SYS                  WRH$_DLM_MISC                            13.0 SYSAUX
SYS                  WRH$_STREAMS_APPLY_SUM_PK                  .1 SYSAUX
SYS                  WRH$_PROCESS_MEM_SUMMARY_PK                .6 SYSAUX
SYS                  WRH$_INST_CACHE_TRANSFER_BL_PK             .1 SYSAUX
SYS                  WRH$_TEMPFILE_PK                           .1 SYSAUX
SYS                  WRH$_EVENT_NAME_PK                         .1 SYSAUX
SYS                  WRH$_LIBRARYCACHE_PK                      2.0 SYSAUX
SYS                  WRH$_BUFFER_POOL_STATS_PK                  .2 SYSAUX
SYS                  WRH$_TABLESPACE_STAT_BL_PK                 .1 SYSAUX
SYS                  WRH$_TS_SPACE_USAGE_IND                    .6 SYSAUX
SYS                  WRH$_SEG_STAT_BL_PK                        .6 SYSAUX
SYS                  WRH$_SEG_STAT_OBJ_PK                      6.0 SYSAUX
SYS                  WRH$_LATCH_CHILDREN_PK                     .1 SYSAUX
SYS                  WRH$_FILESTATXS_PK                        3.4 SYSAUX
SYS                  WRH$_INST_CACHE_TRANSFER_BL                .1 SYSAUX
SYS                  WRH$_SQLSTAT_BL                           2.0 SYSAUX
SYS                  WRH$_SQLTEXT                             16.0 SYSAUX
SYS                  WRH$_LATCH_NAME                            .1 SYSAUX
SYS                  WRH$_LATCH_PARENT_BL                       .1 SYSAUX
SYS                  WRH$_ROWCACHE_SUMMARY_BL                   .3 SYSAUX
SYS                  WRH$_PGASTAT                              3.0 SYSAUX
SYS                  WRH$_STAT_NAME                             .1 SYSAUX
SYS                  WRH$_WAITCLASSMETRIC_HISTORY              9.0 SYSAUX
SYS                  WRH$_ACTIVE_SESSION_HISTORY_BL            8.0 SYSAUX
SYS                  WRH$_LOG                                  3.0 SYSAUX
SYS                  WRH$_SERVICE_WAIT_CLASS_BL                 .3 SYSAUX
SYS                  WRH$_INST_CACHE_TRANSFER                  2.5 SYSAUX
SYS                  WRH$_RULE_SET_PK                           .3 SYSAUX
SYS                  WRH$_STREAMS_POOL_ADVICE_PK               2.0 SYSAUX
SYS                  WRH$_FILESTATXS_BL_PK                      .1 SYSAUX
SYS                  WRH$_LATCH_BL_PK                           .9 SYSAUX
SYS                  WRH$_LATCH_MISSES_SUMRY_BL_PK             2.0 SYSAUX
SYS                  WRH$_PGASTAT_PK                           4.0 SYSAUX
SYS                  WRH$_INSTANCE_RECOVERY_PK                  .2 SYSAUX
SYS                  WRH$_STAT_NAME_PK                          .1 SYSAUX
SYS                  WRH$_PARAMETER_NAME_PK                     .1 SYSAUX
SYS                  WRH$_UNDOSTAT_PK                          1.0 SYSAUX
SYS                  WRH$_DLM_MISC_BL_PK                        .2 SYSAUX
SYS                  WRH$_SERVICE_STAT_BL_PK                    .6 SYSAUX
SYS                  WRH$_SQLSTAT_BL_INDEX                      .3 SYSAUX
SYS                  WRH$_WAITSTAT_PK                          2.9 SYSAUX
SYS                  WRH$_LATCH_MISSES_SUMMARY_PK             51.1 SYSAUX
SYS                  WRH$_ROWCACHE_SUMMARY_PK                  6.2 SYSAUX
SYS                  WRH$_SQLSTAT_PK                          29.9 SYSAUX
SYS                  WRH$_INST_CACHE_TRANSFER_PK               1.3 SYSAUX
SYS                  WRH$_STREAMS_APPLY_SUM                     .1 SYSAUX
SYS                  WRH$_TEMPSTATXS                            .6 SYSAUX
SYS                  WRH$_ENQUEUE_STAT                        10.0 SYSAUX
SYS                  WRH$_SYSSTAT_BL                            .9 SYSAUX
SYS                  WRH$_UNDOSTAT                             3.0 SYSAUX
SYS                  WRH$_SEG_STAT_OBJ                        11.0 SYSAUX
SYS                  WRH$_DLM_MISC_BL                           .4 SYSAUX
SYS                  WRH$_TABLESPACE_SPACE_USAGE               3.0 SYSAUX
SYS                  WRH$_LATCH                               55.1 SYSAUX
SYS                  WRH$_LATCH_PARENT                          .1 SYSAUX
SYS                  WRH$_ROWCACHE_SUMMARY                     7.8 SYSAUX
SYS                  WRH$_LATCH_PARENT_BL_PK                    .1 SYSAUX
SYS                  WRH$_SERVICE_NAME_PK                       .1 SYSAUX
SYS                  WRH$_SERVICE_STAT_PK                     28.1 SYSAUX
SYS                  WRH$_OSSTAT_PK                            2.4 SYSAUX
SYS                  WRH$_SEG_STAT_PK                         19.9 SYSAUX
SYS                  WRH$_STREAMS_POOL_ADVICE                  3.0 SYSAUX
SYS                  WRH$_DATAFILE                              .1 SYSAUX
SYS                  WRH$_TEMPFILE                              .1 SYSAUX
SYS                  WRH$_EVENT_NAME                            .1 SYSAUX
SYS                  WRH$_LATCH_CHILDREN_BL                     .1 SYSAUX
SYS                  WRH$_DB_CACHE_ADVICE_BL                    .2 SYSAUX
SYS                  WRH$_SGA                                   .4 SYSAUX
SYS                  WRH$_SHARED_POOL_ADVICE                   4.0 SYSAUX
SYS                  WRH$_PGA_TARGET_ADVICE                    3.0 SYSAUX
SYS                  WRH$_INSTANCE_RECOVERY                     .3 SYSAUX
SYS                  WRH$_JAVA_POOL_ADVICE                      .9 SYSAUX
SYS                  WRH$_SYSMETRIC_HISTORY                    6.0 SYSAUX
SYS                  WRH$_SYSMETRIC_SUMMARY                   22.0 SYSAUX
SYS                  WRH$_SERVICE_STAT_BL                       .4 SYSAUX
SYS                  WRH$_WAITSTAT                             2.8 SYSAUX
SYS                  WRH$_SGASTAT                              3.4 SYSAUX
SYS                  WRH$_SGA_TARGET_ADVICE_PK                  .9 SYSAUX
SYS                  WRH$_SESS_TIME_STATS_PK                    .3 SYSAUX
SYS                  WRH$_TEMPSTATXS_PK                         .3 SYSAUX
SYS                  WRH$_PARAMETER_BL_PK                       .7 SYSAUX
SYS                  WRH$_FILEMETRIC_HISTORY_INDEX              .1 SYSAUX
SYS                  WRH$_PARAMETER_PK                        39.1 SYSAUX
SYS                  WRH$_TABLESPACE_STAT_PK                   2.4 SYSAUX
SYS                  WRH$_SERVICE_WAIT_CLASS_PK                5.1 SYSAUX
SYS                  WRH$_SQLSTAT_INDEX                       19.0 SYSAUX
                                                    --------------
sum                                                        1,779.2
 
189 rows selected.
 
SQL> select 1779.2/1841 Megs from dual; -- Megs per Snapshot
 
       MEGS
-----------
 .966431287
 
SQL> select 7*24 Snaps from dual; -- Snaps in a week
 
     SNAPS
----------
       168
 
SQL> select 30*24 Snaps from dual; -- Snaps in a 30 days
 
     SNAPS
----------
       720
 
SQL> select 365*24 Snaps from dual; -- snaps in a year
 
     SNAPS
----------
      8760
 
SQL> -- With AWR Rentention set to 7 years. So that's 
 
SQL> select (7*8760*.966431287)/1024 Gigs from dual; 
 
      GIGS
----------
57.8726236
 
SQL> select 57.87*3 Gigs from dual; -- and there are 3 nodes in his RAC environment
 
      GIGS
----------
    173.61
 
SQL> -- So that's 174G for AWR data for 7 years with 3 nodes and one hour snapshots!

So that’s a lot of space to keep a history of parameter changes (although having all that AWR history could be very nice for other uses as well). Another thought that crossed my mind was to create a trigger on the AWR table that holds the parameter information and copy just the changes off to another table. Seems like a very usable approach. Unfortunately, unlike its predecessor Statspack, AWR’s tables are owned by SYS, so no triggers allowed. I guess we could schedule a job to make a copy every hour, but I will leave that as an exercise for the reader. If you do that, you will probably want to do the analytics in the move so you’re not keeping much data in the history table, as opposed to moving it all and doing the lag function in the report. Anyway, that’s enough for one post. Let me know if you have any questions.

Update (12/18/09) : As Doug Burns mentions in his follow up post on his own blog (see his comment below), Oracle provides a nice script for estimating SYSAUX space usage by AWR. Here’s a sample run of the script (it reports current usage as well as an estimated size based on several variables that the user can input).

^?View Code NONE

 
 
SYS@LAB1024> @$ORACLE_HOME/rdbms/admin/utlsyxsz.sql
 
 
This script estimates the space required for the SYSAUX tablespace.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
Specify the Report File Name
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The default report file name is utlsyxsz.txt.  To use this name,
press  to continue, otherwise enter an alternative.
 
Enter value for report_name: 
 
Using the report name utlsyxsz.txt
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SYSAUX Size Estimation Report
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
Estimated at
16:01:00 on Dec 18, 2009 ( Friday ) in Timezone -06:00
 
 
DB_NAME     HOST_PLATFORM                             INST STARTUP_TIME      PAR
----------- ---------------------------------------- ----- ----------------- ---
* LAB1024   homer - Linux IA (32-bit)                    1 11:31:19 (11/20)  NO
 
~~~~~~~~~~~~~~~~~~~~
Current SYSAUX usage
~~~~~~~~~~~~~~~~~~~~
| Total SYSAUX size:                         489.9 MB
|
| Total size of SM/AWR                       108.0 MB (  22.0% of SYSAUX )
| Total size of SM/OPTSTAT                    79.9 MB (  16.3% of SYSAUX )
| Total size of EM                            48.8 MB (  10.0% of SYSAUX )
| Total size of XDB                           48.2 MB (   9.8% of SYSAUX )
| Total size of SDO                           32.9 MB (   6.7% of SYSAUX )
| Total size of XSOQHIST                      23.9 MB (   4.9% of SYSAUX )
| Total size of AO                            23.9 MB (   4.9% of SYSAUX )
| Total size of XSAMD                         15.6 MB (   3.2% of SYSAUX )
| Total size of SM/ADVISOR                    10.4 MB (   2.1% of SYSAUX )
| Total size of WM                             7.1 MB (   1.5% of SYSAUX )
| Total size of LOGMNR                         5.9 MB (   1.2% of SYSAUX )
| Total size of SM/OTHER                       4.9 MB (   1.0% of SYSAUX )
| Total size of TEXT                           4.6 MB (   0.9% of SYSAUX )
| Total size of EXPRESSION_FILTER              3.6 MB (   0.7% of SYSAUX )
| Total size of EM_MONITORING_USER             1.6 MB (   0.3% of SYSAUX )
| Total size of LOGSTDBY                       0.9 MB (   0.2% of SYSAUX )
| Total size of JOB_SCHEDULER                  0.8 MB (   0.2% of SYSAUX )
| Total size of STREAMS                        0.5 MB (   0.1% of SYSAUX )
| Total size of ORDIM                          0.5 MB (   0.1% of SYSAUX )
| Total size of ODM                            0.3 MB (   0.1% of SYSAUX )
| Total size of TSM                            0.3 MB (   0.1% of SYSAUX )
| Total size of Others                        67.6 MB (  13.8% of SYSAUX )
|
 
~~~~~~~~~~~~~~~~~~~~
AWR Space Estimation
~~~~~~~~~~~~~~~~~~~~
 
| To estimate the size of the Automatic Workload Repository (AWR)
| in SYSAUX, we need the following values:
|
|     - Interval Setting (minutes)
|     - Retention Setting (days)
|     - Number of Instances
|     - Average Number of Active Sessions
|     - Number of Datafiles
 
|
| For 'Interval Setting',
|   Press  to use the current value:     60.0 minutes
|   otherwise enter an alternative
|
Enter value for interval: 
 
**   Value for 'Interval Setting': 60
 
|
| For 'Retention Setting',
|   Press  to use the current value:   7.00 days
|   otherwise enter an alternative
|
Enter value for retention: 
 
**   Value for 'Retention Setting': 7
 
|
| For 'Number of Instances',
|   Press  to use the current value:   1.00
|   otherwise enter an alternative
|
Enter value for num_instances: 
 
**   Value for 'Number of Instances': 1
 
|
| For 'Average Number of Active Sessions',
|   Press  to use the current value:   0.00
|   otherwise enter an alternative
|
Enter value for active_sessions: 50
 
**   Value for 'Average Number of Active Sessions': 50
 
| ***************************************************
| Estimated size of AWR:                     830.2 MB
|
|   The AWR estimate was computed using
|   the following values:
|
|            Interval -        60 minutes
|           Retention -      7.00 days
|       Num Instances -         1
|     Active Sessions -     50.00
|           Datafiles -         5
| ***************************************************
 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Optimizer Stat History Space Estimation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
| To estimate the size of the Optimizer Statistics History
| we need the following values:
|
|     - Number of Tables in the Database
|     - Number of Partitions in the Database
|     - Statistics Retention Period (days)
|     - DML Activity in the Database (level)
 
|
| For 'Number of Tables',
|   Press  to use the current value:    274.0
|   otherwise enter an alternative 
|
Enter value for number_of_tables: 
 
**   Value for 'Number of Tables': 274
 
|
| For 'Number of Partitions',
|   Press  to use the current value:   0.00
|   otherwise enter an alternative 
|
Enter value for number_of_partitions: 
 
**   Value for 'Number of Partitions': 0
 
|
| For 'Statistics Retention',
|   Press  to use the current value:     31.0 days
|   otherwise enter an alternative 
|
Enter value for stats_retention: 
 
**   Value for 'Statistics Retention': 31
 
|
| For 'DML Activity',
|   Press  to use the current value:        2 
|   otherwise enter an alternative <1=low, 2=medium, 3=high>
|
Enter value for dml_activity: 
 
**   Value for 'DML Activity': 2
 
| ***************************************************
| Estimated size of Stats history             41.2 MB
|
|   The space for Optimizer Statistics history was
|   estimated using the following values:
|
|                         Tables -     274
|                        Indexes -     615
|                        Columns -   4,126
|                     Partitions -       0
|          Indexes on Partitions -       0
|          Columns in Partitions -       0
|        Stats Retention in Days -      31
|          Level of DML Activity -  Medium
| ***************************************************
 
~~~~~~~~~~~~~~~~~~~~~~
Estimated SYSAUX usage
~~~~~~~~~~~~~~~~~~~~~~
 
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Estimated size of AWR:                     830.2 MB
|
|   The AWR estimate was computed using
|   the following values:
|
|            Interval -        60 minutes
|           Retention -      7.00 days
|       Num Instances -         1
|     Active Sessions -     50.00
|           Datafiles -         5
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Estimated size of Stats history             41.2 MB
|
|   The space for Optimizer Statistics history was
|   estimated using the following values:
|
|                         Tables -     274
|                        Indexes -     615
|                        Columns -   4,126
|                     Partitions -       0
|          Indexes on Partitions -       0
|          Columns in Partitions -       0
|        Stats Retention in Days -      31
|          Level of DML Activity -  Medium
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|   For all the other components, the estimate
|   is equal to the current space usage of
|   the component.
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
| ***************************************************
| Summary of SYSAUX Space Estimation
| ***************************************************
 
| Est size of EM                              48.8 MB
| Est size of XDB                             48.2 MB
| Est size of SDO                             32.9 MB
| Est size of XSOQHIST                        23.9 MB
| Est size of AO                              23.9 MB
| Est size of XSAMD                           15.6 MB
| Est size of SM/ADVISOR                      10.4 MB
| Est size of WM                               7.1 MB
| Est size of LOGMNR                           5.9 MB
| Est size of SM/OTHER                         4.9 MB
| Est size of TEXT                             4.6 MB
| Est size of EXPRESSION_FILTER                3.6 MB
| Est size of EM_MONITORING_USER               1.6 MB
| Est size of LOGSTDBY                         0.9 MB
| Est size of JOB_SCHEDULER                    0.8 MB
| Est size of STREAMS                          0.5 MB
| Est size of ORDIM                            0.5 MB
| Est size of ODM                              0.3 MB
| Est size of TSM                              0.3 MB
| Est size of Others                          67.6 MB
 
| Est size of SM/AWR                         830.2 MB
| Est size of SM/OPTSTAT                      41.2 MB
|
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Total Estimated SYSAUX size:             1,173.5 MB
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| ***************************************************
 
End of Report
SYS@LAB1024>
SYS@LAB1024> -- Let's compare the scripts output to a sum of size for the underlying tables
SYS@LAB1024>
SYS@LAB1024> @table_size
Enter value for owner: 
Enter value for table_name: WRH$%
 
OWNER                SEGMENT_NAME                   TOTALSIZE_MEGS TABLESPACE_NAME
-------------------- ------------------------------ -------------- ------------------------------
SYS                  WRH$_LATCH_MISSES_SUMMARY_BL               .1 SYSAUX
SYS                  WRH$_DB_CACHE_ADVICE_BL_PK                 .1 SYSAUX
SYS                  WRH$_SGASTAT                               .7 SYSAUX
SYS                  WRH$_PROCESS_MEMORY_SUMMARY                .1 SYSAUX
SYS                  WRH$_STREAMS_POOL_ADVICE_PK                .3 SYSAUX
SYS                  WRH$_SQL_WORKAREA_HISTOGRAM                .3 SYSAUX
SYS                  WRH$_SQL_WORKAREA_HIST_PK                  .3 SYSAUX
SYS                  WRH$_PGA_TARGET_ADVICE                     .3 SYSAUX
SYS                  WRH$_PARAMETER_BL_PK                       .2 SYSAUX
SYS                  WRH$_UNDOSTAT                              .2 SYSAUX
SYS                  WRH$_SEG_STAT_BL_PK                        .1 SYSAUX
SYS                  WRH$_SYSMETRIC_SUMMARY_INDEX              2.0 SYSAUX
SYS                  WRH$_DLM_MISC_BL_PK                        .1 SYSAUX
SYS                  WRH$_ACTIVE_SESSION_HISTORY_BL             .1 SYSAUX
SYS                  WRH$_LOG                                   .1 SYSAUX
SYS                  WRH$_LOG_PK                                .1 SYSAUX
SYS                  WRH$_INST_CACHE_TRANSFER                   .1 SYSAUX
SYS                  WRH$_SESS_TIME_STATS_PK                    .1 SYSAUX
SYS                  WRH$_STREAMS_CAPTURE_PK                    .1 SYSAUX
SYS                  WRH$_STREAMS_APPLY_SUM                     .1 SYSAUX
SYS                  WRH$_STREAMS_APPLY_SUM_PK                  .1 SYSAUX
SYS                  WRH$_BUFFERED_SUBSCRIBERS_PK               .1 SYSAUX
SYS                  WRH$_TEMPSTATXS_PK                         .1 SYSAUX
SYS                  WRH$_FILESTATXS                            .6 SYSAUX
SYS                  WRH$_SQLSTAT_BL                            .4 SYSAUX
SYS                  WRH$_SQLTEXT_PK                            .1 SYSAUX
SYS                  WRH$_SQL_SUMMARY_PK                        .1 SYSAUX
SYS                  WRH$_SQL_PLAN_PK                          2.0 SYSAUX
SYS                  WRH$_SQL_BIND_METADATA_PK                  .4 SYSAUX
SYS                  WRH$_LATCH_NAME                            .1 SYSAUX
SYS                  WRH$_ENQUEUE_STAT_PK                       .9 SYSAUX
SYS                  WRH$_DB_CACHE_ADVICE                       .7 SYSAUX
SYS                  WRH$_SGA_PK                                .1 SYSAUX
SYS                  WRH$_SYSSTAT_BL                            .3 SYSAUX
SYS                  WRH$_OSSTAT_BL                             .1 SYSAUX
SYS                  WRH$_OSSTAT_BL_PK                          .1 SYSAUX
SYS                  WRH$_SERVICE_STAT                         1.5 SYSAUX
SYS                  WRH$_CURRENT_BLOCK_SERVER                  .1 SYSAUX
SYS                  WRH$_OSSTAT                                .6 SYSAUX
SYS                  WRH$_RULE_SET                              .1 SYSAUX
SYS                  WRH$_LATCH_CHILDREN_PK                     .1 SYSAUX
SYS                  WRH$_SERVICE_STAT_PK                      2.8 SYSAUX
SYS                  WRH$_TEMPFILE                              .1 SYSAUX
SYS                  WRH$_COMP_IOSTAT_PK                        .1 SYSAUX
SYS                  WRH$_SQLSTAT_BL_PK                         .1 SYSAUX
SYS                  WRH$_SQLSTAT_BL_INDEX                      .1 SYSAUX
SYS                  WRH$_SQL_PLAN                             6.0 SYSAUX
SYS                  WRH$_OPTIMIZER_ENV                         .1 SYSAUX
SYS                  WRH$_OPTIMIZER_ENV_PK                      .1 SYSAUX
SYS                  WRH$_WAITSTAT                              .7 SYSAUX
SYS                  WRH$_EVENT_NAME                            .1 SYSAUX
SYS                  WRH$_EVENT_NAME_PK                         .1 SYSAUX
SYS                  WRH$_BG_EVENT_SUMMARY                      .3 SYSAUX
SYS                  WRH$_LATCH                                6.6 SYSAUX
SYS                  WRH$_WAITSTAT_BL                           .1 SYSAUX
SYS                  WRH$_SYSTEM_EVENT_PK                      1.3 SYSAUX
SYS                  WRH$_LATCH_MISSES_SUMRY_BL_PK              .1 SYSAUX
SYS                  WRH$_ROWCACHE_SUMMARY_BL_PK                .1 SYSAUX
SYS                  WRH$_SGA                                   .1 SYSAUX
SYS                  WRH$_SGA_TARGET_ADVICE                     .1 SYSAUX
SYS                  WRH$_PARAMETER_NAME                        .1 SYSAUX
SYS                  WRH$_METRIC_NAME_PK                        .1 SYSAUX
SYS                  WRH$_SYSMETRIC_SUMMARY                    3.0 SYSAUX
SYS                  WRH$_DB_CACHE_ADVICE_PK                    .7 SYSAUX
SYS                  WRH$_TABLESPACE_STAT_BL_PK                 .1 SYSAUX
SYS                  WRH$_LATCH_PARENT_PK                       .1 SYSAUX
SYS                  WRH$_ACTIVE_SESSION_HISTORY_PK             .7 SYSAUX
SYS                  WRH$_FILESTATXS_BL                         .1 SYSAUX
SYS                  WRH$_DATAFILE                              .1 SYSAUX
SYS                  WRH$_TEMPFILE_PK                           .1 SYSAUX
SYS                  WRH$_SYSTEM_EVENT_BL                       .1 SYSAUX
SYS                  WRH$_SYSTEM_EVENT_BL_PK                    .1 SYSAUX
SYS                  WRH$_LIBRARYCACHE                          .3 SYSAUX
SYS                  WRH$_SGASTAT_BL                            .1 SYSAUX
SYS                  WRH$_PGASTAT_PK                            .4 SYSAUX
SYS                  WRH$_PGA_TARGET_ADVICE_PK                  .2 SYSAUX
SYS                  WRH$_SGA_TARGET_ADVICE_PK                  .2 SYSAUX
SYS                  WRH$_INSTANCE_RECOVERY_PK                  .1 SYSAUX
SYS                  WRH$_LATCH_PK                             4.1 SYSAUX
SYS                  WRH$_SYS_TIME_MODEL_BL_PK                  .1 SYSAUX
SYS                  WRH$_UNDOSTAT_PK                           .1 SYSAUX
SYS                  WRH$_SYSMETRIC_HISTORY                     .1 SYSAUX
SYS                  WRH$_SESSMETRIC_HISTORY_INDEX              .1 SYSAUX
SYS                  WRH$_STREAMS_CAPTURE                       .1 SYSAUX
SYS                  WRH$_RULE_SET_PK                           .1 SYSAUX
SYS                  WRH$_FILESTATXS_PK                         .6 SYSAUX
SYS                  WRH$_LATCH_NAME_PK                         .1 SYSAUX
SYS                  WRH$_BG_EVENT_SUMMARY_PK                   .3 SYSAUX
SYS                  WRH$_WAITSTAT_BL_PK                        .1 SYSAUX
SYS                  WRH$_LATCH_PARENT                          .1 SYSAUX
SYS                  WRH$_DB_CACHE_ADVICE_BL                    .1 SYSAUX
SYS                  WRH$_SGASTAT_BL_U                          .1 SYSAUX
SYS                  WRH$_RESOURCE_LIMIT                        .1 SYSAUX
SYS                  WRH$_SHARED_POOL_ADVICE                    .4 SYSAUX
SYS                  WRH$_STAT_NAME                             .1 SYSAUX
SYS                  WRH$_STAT_NAME_PK                          .1 SYSAUX
SYS                  WRH$_OSSTAT_NAME_PK                        .1 SYSAUX
SYS                  WRH$_PARAMETER_NAME_PK                     .1 SYSAUX
SYS                  WRH$_SEG_STAT_OBJ_PK                       .4 SYSAUX
SYS                  WRH$_METRIC_NAME                           .1 SYSAUX
SYS                  WRH$_FILEMETRIC_HISTORY                    .1 SYSAUX
SYS                  WRH$_CR_BLOCK_SERVER                       .1 SYSAUX
SYS                  WRH$_TABLESPACE_SPACE_USAGE                .1 SYSAUX
SYS                  WRH$_TS_SPACE_USAGE_IND                    .1 SYSAUX
SYS                  WRH$_SERVICE_NAME                          .1 SYSAUX
SYS                  WRH$_SERVICE_NAME_PK                       .1 SYSAUX
SYS                  WRH$_SERVICE_WAIT_CLASS_BL                 .1 SYSAUX
SYS                  WRH$_BUFFERED_QUEUES                       .1 SYSAUX
SYS                  WRH$_BUFFERED_QUEUES_PK                    .1 SYSAUX
SYS                  WRH$_SYS_TIME_MODEL_PK                     .7 SYSAUX
SYS                  WRH$_SQLTEXT                              2.0 SYSAUX
SYS                  WRH$_SQL_SUMMARY                           .1 SYSAUX
SYS                  WRH$_LATCH_BL                              .4 SYSAUX
SYS                  WRH$_LATCH_BL_PK                           .3 SYSAUX
SYS                  WRH$_LATCH_CHILDREN_BL                     .1 SYSAUX
SYS                  WRH$_LATCH_CHILDREN_BL_PK                  .1 SYSAUX
SYS                  WRH$_LATCH_MISSES_SUMMARY                 1.6 SYSAUX
SYS                  WRH$_LATCH_PARENT_BL_PK                    .1 SYSAUX
SYS                  WRH$_ROWCACHE_SUMMARY_BL                   .1 SYSAUX
SYS                  WRH$_JAVA_POOL_ADVICE                      .1 SYSAUX
SYS                  WRH$_JAVA_POOL_ADVICE_PK                   .1 SYSAUX
SYS                  WRH$_PARAMETER                            3.6 SYSAUX
SYS                  WRH$_SYSSTAT_BL_PK                         .3 SYSAUX
SYS                  WRH$_SYS_TIME_MODEL_BL                     .1 SYSAUX
SYS                  WRH$_LATCH_MISSES_SUMMARY_PK              1.6 SYSAUX
SYS                  WRH$_SEG_STAT_OBJ                          .8 SYSAUX
SYS                  WRH$_WAITCLASSMETRIC_HIST_IND             2.0 SYSAUX
SYS                  WRH$_DLM_MISC_BL                           .1 SYSAUX
SYS                  WRH$_CURRENT_BLOCK_SERVER_PK               .1 SYSAUX
SYS                  WRH$_INST_CACHE_TRANSFER_BL                .1 SYSAUX
SYS                  WRH$_TABLESPACE_STAT                       .6 SYSAUX
SYS                  WRH$_MTTR_TARGET_ADVICE                    .1 SYSAUX
SYS                  WRH$_SERVICE_STAT_BL                       .1 SYSAUX
SYS                  WRH$_SESS_TIME_STATS                       .1 SYSAUX
SYS                  WRH$_BUFFERED_SUBSCRIBERS                  .1 SYSAUX
SYS                  WRH$_SGASTAT_U                             .7 SYSAUX
SYS                  WRH$_SYSSTAT_PK                           4.0 SYSAUX
SYS                  WRH$_TABLESPACE_STAT_PK                    .6 SYSAUX
SYS                  WRH$_SERVICE_WAIT_CLASS_PK                 .7 SYSAUX
SYS                  WRH$_FILESTATXS_BL_PK                      .1 SYSAUX
SYS                  WRH$_TEMPSTATXS                            .1 SYSAUX
SYS                  WRH$_COMP_IOSTAT                           .1 SYSAUX
SYS                  WRH$_SQL_BIND_METADATA                     .4 SYSAUX
SYS                  WRH$_LATCH_CHILDREN                        .1 SYSAUX
SYS                  WRH$_LATCH_PARENT_BL                       .1 SYSAUX
SYS                  WRH$_LIBRARYCACHE_PK                       .3 SYSAUX
SYS                  WRH$_ROWCACHE_SUMMARY                     1.3 SYSAUX
SYS                  WRH$_BUFFER_POOL_STATISTICS                .1 SYSAUX
SYS                  WRH$_BUFFER_POOL_STATS_PK                  .1 SYSAUX
SYS                  WRH$_WAITSTAT_PK                           .7 SYSAUX
SYS                  WRH$_PROCESS_MEM_SUMMARY_PK                .1 SYSAUX
SYS                  WRH$_SHARED_POOL_ADVICE_PK                 .2 SYSAUX
SYS                  WRH$_STREAMS_POOL_ADVICE                   .3 SYSAUX
SYS                  WRH$_SERVICE_WAIT_CLASS                    .7 SYSAUX
SYS                  WRH$_SYSSTAT                              3.5 SYSAUX
SYS                  WRH$_PARAMETER_BL                          .3 SYSAUX
SYS                  WRH$_OSSTAT_NAME                           .1 SYSAUX
SYS                  WRH$_SEG_STAT_BL                           .2 SYSAUX
SYS                  WRH$_SESSMETRIC_HISTORY                    .1 SYSAUX
SYS                  WRH$_WAITCLASSMETRIC_HISTORY              2.0 SYSAUX
SYS                  WRH$_CR_BLOCK_SERVER_PK                    .1 SYSAUX
SYS                  WRH$_ASH_BL_PK                             .1 SYSAUX
SYS                  WRH$_TABLESPACE_STAT_BL                    .1 SYSAUX
SYS                  WRH$_ACTIVE_SESSION_HISTORY                .9 SYSAUX
SYS                  WRH$_SERVICE_WAIT_CLASS_BL_PK              .1 SYSAUX
SYS                  WRH$_SEG_STAT_PK                          1.3 SYSAUX
SYS                  WRH$_OSSTAT_PK                             .6 SYSAUX
SYS                  WRH$_SQLSTAT                              6.0 SYSAUX
SYS                  WRH$_DATAFILE_PK                           .1 SYSAUX
SYS                  WRH$_SYSTEM_EVENT                         1.4 SYSAUX
SYS                  WRH$_SQLSTAT_PK                           2.6 SYSAUX
SYS                  WRH$_SQLSTAT_INDEX                        1.6 SYSAUX
SYS                  WRH$_ENQUEUE_STAT                         2.0 SYSAUX
SYS                  WRH$_PGASTAT                               .3 SYSAUX
SYS                  WRH$_RESOURCE_LIMIT_PK                     .2 SYSAUX
SYS                  WRH$_INSTANCE_RECOVERY                     .1 SYSAUX
SYS                  WRH$_THREAD                                .1 SYSAUX
SYS                  WRH$_THREAD_PK                             .1 SYSAUX
SYS                  WRH$_SYS_TIME_MODEL                        .7 SYSAUX
SYS                  WRH$_SEG_STAT                             2.6 SYSAUX
SYS                  WRH$_DLM_MISC                              .1 SYSAUX
SYS                  WRH$_SYSMETRIC_HISTORY_INDEX               .1 SYSAUX
SYS                  WRH$_FILEMETRIC_HISTORY_INDEX              .1 SYSAUX
SYS                  WRH$_INST_CACHE_TRANSFER_BL_PK             .1 SYSAUX
SYS                  WRH$_ROWCACHE_SUMMARY_PK                  1.3 SYSAUX
SYS                  WRH$_SERVICE_STAT_BL_PK                    .1 SYSAUX
SYS                  WRH$_PARAMETER_PK                         2.8 SYSAUX
SYS                  WRH$_DLM_MISC_PK                           .1 SYSAUX
SYS                  WRH$_INST_CACHE_TRANSFER_PK                .1 SYSAUX
                                                    --------------
sum                                                          103.1

Note that the script reports 5M more than my sum of table and index sizes (108M vs 103M). Close enough for government work!