The CPI seems to be an issue on the list now. This note is in response to the postings of Doug and Marianne (found below my response). Pls refer to my previous posts to Eric and Michael for further information. DISCLAIMER: What follows are my own thoughts and not those of BLS. I personally have been working these issues for several years, but the numbers presented here and the state of various projects are off the top of my head. I did not specifically check the latest official bias estimates nor did I check on the status of any particular project as I wrote this. The answer to Doug's question, why do we continue to put out misleading numbers, is literally that we don't know any better. On the other hand, the problems below have by and large been discovered internally and are being worked on internally: we do not take kindly to people telling us that we do not know what we are doing on the basis of our own research. The bias in the CPI comes from five sources: 1. Initiation bias. The CPI attempts to be a Laspeyres Index, a price index with weights equal to beginning period quantities. However, we do not ever collect quantities but instead collect expenditures. When an item is first used in the index the weight is determined by dividing the expenditure by the price at that time. The result is that items which are on sale get too high a weight while for those not on sale the weight is too low. Now, the probability of sale items (high weights) going up is much higher than the probability of the regular price items (low weights) going up, and hence the index has an upward bias for the first few months after a new item is initiated. The magnitude of this bias for the all-items index is perhaps 0.4% per year. The current procedure which imparts this bias has been in place only since 1978 and hence the earlier numbers are free of it. While the reason for the bias is not complex it nevertheless was quite challenging to figure it out in practice since the actual procedure for calculating the index is quite complicated. We were alerted to the problem by anomalous results comparing average prices with the index for certain items. Our task was not made any easier by the presence of disinformation in the form of a superficially plausible but quite wrong explanation for the discrepancy. The name that has come to be applied to this problem is "formula bias," and we will eliminate it beginning in Jan., 1997. 2. Substitution bias. The Laspeyres Index is based on pricing the same quantities each month while consumers vary their spending patterns to respond to changing prices. Thus theoretically the Laspeyres Index under certain conditions provides an upper bound to the Konus (constant utility) Index. The magnitude of this bias is 0.2-0.3% per year. To remedy this problem by brute force would require collecting quantities as well as prices every month, an approach that is precluded by budget considerations. I am at present working to develop procedure to provide at least an approximate solution econometrically. Are you holding your breath waiting for it? Please don't. 3. New product (technological change) bias. All index formulas that I am familiar with incorrectly assume N goods which are available each month, and hence new products are ruled out by assumption. This is very harmful since there is sometimes great utility from a new product, and a price index misses this entirely. In addition, there is often a life cycle pricing pattern as the price starts high, falls to a "normal" level as the product matures, and may rise again at the end. An index will tend to miss the decrease and overweight the later increase. The magnitude of this effect is perhaps 0.5% per year. We are making some steps to correct this by initiating certain items, e.g., pharmaceuticals and electronics, more often, but a thorough attempt to capture this effect could require enormous resources. 4. Outlet substitution bias. We do not capture the savings arising as shoppers shift away from high priced stores to lower priced ones. The magnitude of this effect is about 0.1% per year. Off hand I do not know of a good way to fix this directly, although I have the very strong feeling that it would take a substantial financial commitment. 5. Small sample bias. The national CPI is calculated as a Laspeyres Index of metropolitan area CPI's. In small samples the expectation of the ratio of weighted averages of prices is not the ratio of the expectations. The magnitude of this effect is about 0.1% per year. The solution would be to change our procedure for calculating the index by replacing the current bottom up procedure with a top down one. This has been rejected at present for fear that this big a computer change would lead to the possibility of a major error in index calculation. The sum of these biases is a little less than 1.5% per year. Leaving out formula bias, which began in 1978, the bias in the golden age was of the order of 1% per year, and hence, to answer Doug's question, growth was even higher than previously reported. I do not know what Janet Norwood thinks on these issues. Marianne notes that WEFA estimates the total bias as 394/367-1 = 7.3% over the 25 years from 1970 to 1995. Not being familiar with the work I cannot comment specifically, but I have the feeling that all of the factors listed above were not considered. Marianne also suggests that we underestimate "the rise in the cost of child care, elder care, meals, laundry as production has shifted out of the informal sector." Off hand I am not sure. What has happened is that these activities have increasingly become market activities as home production has shrunk. Thus to compare them would require the imputation of a wage rate to the home production and a whole host of complicated side calculations. I do not know the direction of the bias that would result. I do suspect, however, that since these activities comprise a rather small share of the national economy, the effect on the CPI would not be great. Marianne's comment into the nonmarket economy leads me to the environment, an element of the cost of living which should get more attention than it does. Indeed, the Konus constant utility concept, taken literally, does require an environmental component. For example, at present we adjust the prices of new cars to reflect new features, and attempt thereby to measure the pure price effect. We do not, however, consider the condition of the roads. Thus there is a basic question which is seldom, if ever, asked. On a 90 degree day, are we better off with the air conditioner on listening to our favorite CD's while sitting in traffic than we were 50 years ago with the windows open listening to an AM radio while poking along at 30 mph? There are lots of these examples, and as far as I know careful estimates of the total effect have not been presented. Dave Richardson ---------- I'm surprised to see someone from the BLS say the CPI is so badly overstated. On what evidence is this based? If the BLS internally thinks this, why does it continue to put out such misleading numbers? Former BLS Commissioner Janet Norwood seems not to agree. And if it's overstated now, was it also so in the past, meaning that real wage growth was even higher during the golden age than officially reported? Doug -- Doug Henwood Left Business Observer 250 W 85 St New York NY 10024-3217 USA +1-212-874-4020 voice +1-212-874-3137 fax email: <[EMAIL PROTECTED]> web: <http://www.panix.com/~dhenwood/LBO_home.html> ----------- The WEFA Group (Wharton Econometrics and Chase Manhattan merged) have been using their own consumer expenditures deflator, which I believes is based on data that goes into the CPI and other US data. While they show a slower rate of inflation than the CPI, the difference is not that great: they get a 367 percent increase between 1970 and 1995 versus the 394 percent increase shown by the CPI. My understanding is that the downward revisions in the CPI, while adjusting for upward biases in the CPI, do not take adequately into account downward biases (eg underestimation of the rise in the cost of child care, elder care, meals, laundry as production has shifted out of the informal sector). Marianne Hill [EMAIL PROTECTED]
