I think you must have cut off the last d in the script. The last line should look like this:

) d;

If that’s not it, post a little more info (i.e. run the script with echo on and capture the output) and I’ll see if I can spot the problem. I haven’t had any issues with it on 10.1 - 11.2.

Kerry

Once a Profile has been created, its hints will be applied (as long as they are valid, and they don’t contradict each other, etc…). Since this was a simple Profile with a single directive (INDEX_RS_ASC), when that hint was found to be invalid, it went back to the Full Table Scan. The reason the hint was invalid is because an index range scan cannot be done on an index where the leading columns are not part of the predicate. A skip scan or a full index scan are possible, but not a normal range scan. And since the IGNORE_OPTIM_EMBEDDED_HINTS hint is also included in the Profile, the inline hint in the text is ignored. So basically it’s back to picking a plan without any directive hints. At least that’s what I believe is happening.

Kerry

a) I agree that Profiles appear to have been built with statistical modifications in mind. It looks to me like they “borrowed” the code from Outlines though. Adding the ability to do the “force matching” thing that let’s them apply to multiple statements that only differ due to literals. My experience with the Profiles generated by SQL Tuning Advisor has not been very good in terms of plan stability though. It does not appear that they have any concept of changes to stats and just apply their scaling to what ever the optimizer comes up with. So if (I should say when) the stats change, the scaling factors may not be correct anymore. I can see them working well for situations where the stats gathering is managed tightly and the optimizer just has no way to deal with a particular situation (like correlated columns or example), but for the general case, they seem to “sour” over time due to changing stats.

Yes, I have seen other hints. We have a client that has created many Profiles with the Tuning Advisor (they are in the process of getting rid of them by the way, due to this tendency to “sour” over time). But back to question about additional hints, I just did a quick review of about 40 Profiles and the vast majority have only statistical hints (OPT_ESTIMATE, COLUMN_STATS, TABLE_STATS, INDEX_STATS). A few have some variation of the (ALL_ROWS, FIRST_ROWS(X)) hint and most have the OPTIMIZER_FEATURES_ENABLE hint as well. To my surprise, I found a couple that had IGNORE_OPTIM_EMBEDDED_HINTS as you were expecting. Even more surprising was that one of the statements had no embedded hints while the other did. (something else to look at I guess) I did find one that had a whole slew of hints. Again, I’m not sure why occasionally we get ones that look like this, but at any rate here’s the list of hints:

OPT_ESTIMATE(@”SEL$5DA710D3″, JOIN, (”E”@”SEL$1″, “B”@”SEL$1″, “A”@”SEL$1″, “X”@”SEL$2″), SCALE_ROWS=135.982493)
IGNORE_OPTIM_EMBEDDED_HINTS
OPT_ESTIMATE(@”SEL$5DA710D3″, JOIN, (”F”@”SEL$1″, “B”@”SEL$1″), SCALE_ROWS=26.34181566)
OPTIMIZER_FEATURES_ENABLE(’10.2.0.4′)
OPT_ESTIMATE(@”SEL$5DA710D3″, INDEX_FILTER, “F”@”SEL$1″, IDX$$_1AA260002, SCALE_ROWS=8.883203639e-06)
OPT_PARAM(’optimizer_index_cost_adj’ 80)
OPT_ESTIMATE(@”SEL$5DA710D3″, JOIN, (”D”@”SEL$1″, “C”@”SEL$1″, “A”@”SEL$1″, “X”@”SEL$2″), SCALE_ROWS=1.308307653)
OPT_PARAM(’optimizer_index_caching’ 60)
OPT_ESTIMATE(@”SEL$5DA710D3″, JOIN, (”E”@”SEL$1″, “D”@”SEL$1″, “C”@”SEL$1″, “B”@”SEL$1″, “A”@”SEL$1″, “Y”@”SEL$2″), SCALE_ROWS=862.9359462)
ALL_ROWS
OPT_ESTIMATE(@”SEL$5DA710D3″, JOIN, (”E”@”SEL$1″, “A”@”SEL$1″), SCALE_ROWS=25.60960842)
OUTLINE_LEAF(@”SEL$5DA710D3″)
OPT_ESTIMATE(@”SEL$5DA710D3″, JOIN, (”C”@”SEL$1″, “A”@”SEL$1″), SCALE_ROWS=7.884506683)
UNNEST(@”SEL$2″)
OPT_ESTIMATE(@”SEL$5DA710D3″, JOIN, (”F”@”SEL$1″, “B”@”SEL$1″, “A”@”SEL$1″), SCALE_ROWS=839.9683673)
OUTLINE(@”SEL$1″)
OPT_ESTIMATE(@”SEL$5DA710D3″, JOIN, (”B”@”SEL$1″, “A”@”SEL$1″), SCALE_ROWS=4.446153275)
OUTLINE(@”SEL$2″)
OPT_ESTIMATE(@”SEL$5DA710D3″, TABLE, “D”@”SEL$1″, SCALE_ROWS=11.39782103)
FULL(@”SEL$5DA710D3″ “A”@”SEL$1″)
OPT_ESTIMATE(@”SEL$5DA710D3″, JOIN, (”E”@”SEL$1″, “D”@”SEL$1″, “C”@”SEL$1″, “A”@”SEL$1″), SCALE_ROWS=259.4309108)
FULL(@”SEL$5DA710D3″ “B”@”SEL$1″)
OPT_ESTIMATE(@”SEL$5DA710D3″, JOIN, (”E”@”SEL$1″, “C”@”SEL$1″, “A”@”SEL$1″), SCALE_ROWS=190.2944942)
FULL(@”SEL$5DA710D3″ “X”@”SEL$2″)
OPT_ESTIMATE(@”SEL$5DA710D3″, JOIN, (”E”@”SEL$1″, “C”@”SEL$1″, “A”@”SEL$1″, “X”@”SEL$2″), SCALE_ROWS=26.52093258)
INDEX(@”SEL$5DA710D3″ “F”@”SEL$1″ (”…”))
INDEX_RS_ASC(@”SEL$5DA710D3″ “C”@”SEL$1″ (”…”))
OPT_ESTIMATE(@”SEL$5DA710D3″, INDEX_SKIP_SCAN, “F”@”SEL$1″, IDX$$_1AA260002, SCALE_ROWS=8.883203639e-06)
INDEX_RS_ASC(@”SEL$5DA710D3″ “E”@”SEL$1″ (”…”))
ALL_ROWS
INDEX(@”SEL$5DA710D3″ “D”@”SEL$1″ (”…”))
LEADING(@”SEL$5DA710D3″ “A”@”SEL$1″ “B”@”SEL$1″ “X”@”SEL$2″ “F”@”SEL$1″ “C”@”SEL$1″ “E”@”SEL$1″ “D”@”SEL$1″)
USE_HASH(@”SEL$5DA710D3″ “B”@”SEL$1″)
USE_HASH(@”SEL$5DA710D3″ “X”@”SEL$2″)
USE_NL(@”SEL$5DA710D3″ “F”@”SEL$1″)
USE_NL(@”SEL$5DA710D3″ “C”@”SEL$1″)
USE_NL(@”SEL$5DA710D3″ “E”@”SEL$1″)
USE_NL(@”SEL$5DA710D3″ “D”@”SEL$1″)
SWAP_JOIN_INPUTS(@”SEL$5DA710D3″ “X”@”SEL$2″)

b) Agreed. I am using Profiles in a way that the designers may not have intended. The argument for using Profiles in this manner would be that Profiles can match multiple statements that differ only by literals, while Outlines can’t. Profiles are a little cleaner as well. No need for a database trigger to enable them for example. Also, I have to admit that the documentation saying Outlines have been deprecated is starting to make me nervous about using them.

c) Looks like sometimes Profiles include IGNORE_OPTIM_EMBEDDED_HINTS and sometimes they don’t. My first thought was maybe the ignore hint was included only when there are embedded hints and it’s smart enough to leave the hint out when there aren’t any. But I found one that didn’t have any embedded hints that still had the ignore hint, so at this point it is unclear to me as to why sometimes it is there and not other times …

d) Yep, my tests showed that the plan can be changed by changing the hints as well, regardless of whether there are any statistical hints there or not. And 11g baselines look to me like enhanced Profiles. Although the Baselines have a plan_hash_value - so they can determine if they got the plan that generated the Baseline to begin with - a giant step forward in my opinion - although I wish they had just saved the plan itself and been done with it. It means they can throw out the whole plan and re-optimize (or pick another baseline), instead of just ignoring a single hint.

On the bug issue, if I understand you correctly you’re saying that creating an Outline also results in the buggy behavior (that being that the plan changes due to the invalid hint). I agree with that observation. Creating an Outline or a Profile in 10g or 11gR1 will change the plan due to this bug. Creating a Baseline in 11gR1 does not change the plan (even though the hint is still messed up) because Baselines are smart enough to know if the plan generated by the hints matches the plan that was originally used to generate the hints. And so, it throws the whole thing out and re-optimizes the statement from scratch.

Thanks for all the thoughtful comments.

Kerry

a) Outlines vs. profiles: I think these can be viewed on two levels. On one hand they are both represented by a set of hints; on the other hand they exist for different reasons. In principle an outline fixes an execution plan so that it can’t change whereas a profile fixes the optimizer’s understanding of the data distribution pattern so that it can still pick the right plan as the data size varies.

(Other than optimizer_features_enable() and ignore_optim_embedded_hints, I’ve not seen anything but statistical hints in a profile generated by the tuning tool - do you have any examples to demonstrate otherwise ?).

b) One aspect of the difference between outlines and profiles is that the profiling information doesn’t get into the outline when the statement runs. A statement with a profile can produce many different execution plans - which means many different outlines: which, I’d say, is an argument against copying an outline into the profile table.

c) disabling embedded hints - I think I was wrong, but I’m not 100% sure. I’ve just run up an example in 11.1.0.6 to demonstrate the point - and realised that the profile included the ‘optim_ignore_embedded_hints’ hint, so perhaps there are cases where this isn’t included and embedded hints are still obeyed. (In fact, since you were copying from the outline, you automatically copied in the optim_ignore_embedded_hints hint anyway - and then the index_rs_asc() hint was ignored because it is invalid in this context (as you suggested), and shouldn’t have been genereated by the optimizer).

d) That’s a valid test that the code for applying profiles doesn’t care what type of hint you’ve put in the table - and it certainly seemed to be true for a couple of examples I’ve just created. It certainly makes sense - especially in view of the fact that sql-baselines are written into the same table in 11g (Why have two different pieces of code when one will do ?)

Regards
Jonathan Lewis

Wow - I had to think about this one for a while (and go back to check my work) - here’s my thoughts:

a) A profile and an outline are different things - roughly speaking an outline lists actions and strategies, a profile supplies statistical corrections to the optimizer.

I see Outlines and Profiles (and Baselines for that matter) as basically the same thing. They apply a set of hints to a statement. The standard (documented) way of generating a SQL Profile is to use the the SQL Tuning Advisor - which does calculate scaling factors to apply to various steps and implements them via the OPT_ESTIMATE, TABLE_STATS, COLUMN_STATS hints (and occasionally it adds some strategic type hints…). But Profiles are nevertheless just applying hints (I think).

b) The code you’ve used to “create a profile from the shared pool” doesn’t, it copies an outline from the shared pool but tells the optimizer it’s a profile.

That comment made me laugh - you’re absolutely right of course! But just to be clear to anyone else that might stumble across this dialog, every plan has a set of hints that Oracle thinks will recreate the plan (stored in the other_xml column of v$sql_plan). These hints are the exact hints that are used when you create an Outline. The code in my create_sql_profile.sql script does exactly what you said, it creates a SQL Profile using that same set of hints that would be in the Outline if you created one for the statement.

c) When you run (or explain) the query with the “profile” in place, the optimizer detects the profile and disables the embedded hints (i.e. the ones in the actual text) and tries to derive an execution plan based on what ought to be statistical content in the profile combined with the basic object stats. But the profile supplies no statistical information.

I believe SQL Profiles apply the hints. I don’t think they expect any specific statistical information (i.e. I don’t think they care whether there are any OPT_ESTIMATE, TABLE_STATS, or COLUMN_STATS hints or not) I could be wrong, but I haven’t seen any situation where a Profile was ignored because it didn’t have statistical information.

As a side note, I don’t think the SQL Profiles that are created by the SQL Tuning Advisor disable embedded hints the way Outlines do. I say this because Outline hints include the IGNORE_OPTIM_EMBEDDED_HINTS hint and SQL Profiles created by the Tuning Advisor do not. And if you think about it, that makes sense. Since the latter are basically only supplying statistical information, disabling embedded hints would not make sense. (Note: I have not done a specific test to prove this to myself)

d) In the absence of statistical information, and after disabling the embedded hints, the optimizer has “used” the profile but produces the original, unhinted, tablescan plan.

I came up with a little test case. See if you think this is a valid test.

1. create a SQL Profile (based on hints from other_xml)
2. check hints - there will be no statistical hints
3. manually modify one of the hints in a way that forces a different plan
4. see if it works

If the plan changes as expected, despite the absence of any statistical hints, then that implies to me that Profiles just apply whatever hints are there. Thoughts?

Kerry

I’d like to suggest a different interpretation of your results.

a) A profile and an outline are different things - roughly speaking an outline lists actions and strategies, a profile supplies statistical corrections to the optimizer.

b) The code you’ve used to “create a profile from the shared pool” doesn’t, it copies an outline from the shared pool but tells the optimizer it’s a profile.

c) When you run (or explain) the query with the “profile” in place, the optimizer detects the profile and disables the embedded hints (i.e. the ones in the actual text) and tries to derive an execution plan based on what ought to be statistical content in the profile combined with the basic object stats. But the profile supplies no statistical information.

d) In the absence of statistical information, and after disabling the embedded hints, the optimizer has “used” the profile but produces the original, unhinted, tablescan plan.

Regards
Jonathan Lewis