pcvnp下载

Here is the follow-up on my previous post on decision making under ambiguity. I stopped halfway through the section on non-Bayesian criteria with non-additive beliefs because the post had become humongous. Now the authors discuss how to determine the set of priors in the maxmin expected utility model. One formulation is called epsilon-contamination. "In this formulation, the decision maker has a probabilistic benchmark in the sense that he believes the probability distribution on the state space is a given distribution, say p. But he's not totally confident about this. The way to model this is to say that the set of priors he has in mind will be a combination between the 'probabilistic benchmark' on one hand and 'anything can happen' on the other hand." 

Hansen and Sargent (2001, 2008) also tackle model uncertainty using ideas inspired from robust control (something dear to my heart, since I studied control theory in college before moving on to operations research). In this robust preferences approach, agents rank acts according to a criterion that uses an estimated distribution p and also the relative entropy or Kullback-Leibler divergence between p and other distributions (capturing the possibility that p is not the right distribution). Hansen and Sargent's model can be interpreted in terms of variational preferences.  

Ellsberg's paradox, which basically says that people prefer situations where they have a known probability of winning rather than situations where the probability is unknown ("the devil they know"), can be explained using an extension of the Rank Dependent Utility to compound lotteries or two-layer expected utility. In this model, the decision maker has a set of priors in mind but also a prior over the set of priors, called a second-order belief, over which we take an expectation of distorted expected utilities.  

Jaffray (1989) "Generalizes the standard expected utility under risk to a framework in which the probabilities of the different states of nature are imperfectly known," belonging to given intervals. The authors explain that "to any set of probability distributions, it is possible to associate is lower envelope, associating to each event its lower probability compatible with the set of distributions." This lower envelope is a capacity (an increasing, non-necessarily additive set function). Then the value associated to a decision uses the minimal and maximal outcomes on events and the Mobius transform of the capacity makes an apparition, measuring the ambiguity of the event, i.e., the gap between the lower probability of this event and the sum of the lower probabilities of its sub-events. Oh, and to add to the fun, there is a pessimism-optimism index.

The paper continues with definitions of ambiguity aversion and complexities that arise from including dynamics. Many researchers have considered updating rules for preferences in maxmin expected utilities. There is also a section documenting the experimental evidence about decision makers' attitude toward ambiguity. Finally, the authors end with references about how these decision criteria have been used in finance and auctions. 

It was a very worthwhile read and I highly recommend it!

This is one of the most interesting papers I've read in a while and it is in the 灯蓝专业, not a journal we in operations research and management science typically look at. Plenty of concepts in this paper are relevant to the field of robust optimization, and of course many of my robust optimization colleagues will have heard of some of those concepts, it still feels that our two communities of management scientists and economists should talk together more often. I really enjoyed reading this paper and recommend that everyone read it too. Anyway, here is a summary.

The authors use the terms 'uncertainty' or 'ambiguity' equivalently, to represent non-probabilized uncertainty, as opposed to risk, which represents 'probabilized' uncertainty (you can be sure that sentence caught my interest, and I was still on the first page.) The authors explain: "We will thus concentrate on situations in which there is too little information to pin down easily probabilistic beliefs (as opposed to risky situations, in which objects of choice - lotteries - are already formulated in terms of probability distributions)." One thing you want to remember if you haven't looked at the economics literature in a while is that they view everything random in terms of lotteries when they can and imagine that the decision maker can choose between different lotteries to express his attitude toward risk. The lotteries typically only have two possible outcomes, and only one in a riskless case. 

The authors readily point out that they had to leave out some recent developments in the field, especially the concept of "unforeseen contingencies and more generally the issue of subjective state space" because it is too early to distinguish the main contributions from the rest. But their paper was first published about ten years ago so I was curious to discover the developments in those fields in 2020. Some articles about unforeseen contingencies include "Unforeseen Contingencies and Incomplete Contracts" by Eric Maskin and economics superstar Jean Tirole in the Review of Economics Studies 66(1): 83-114, 1999, who argue that what matters is that "agents can probabilistically forecast their possible future payoffs (even if other aspects of the state of the nature cannot be forecast). In other words, all that is required for optimality is that agents be able to perform dynamic programming, an assumption always invoked by the incomplete contract literature". The ability to perform dynamic programming is certainly a valuable skill for anyone to have in life. In "Static Choice in the Presence of Unforeseen Contingencies," by David M. Kreps, in the book "Economic Analysis of Markets and Games", Kreps presents a way to "model conceptually the idea that certain contingencies may be unforeseen", which gives rise to "an implicit representation theorem that is remarkably similar to the Savage model" - the 1954 model at the core of choice under uncertainty.

Savage developed subjective expected utility theory and characterized individual choice in presence of uncertainty as expected utility maximizing behavior. The most important axiom of his theory is known as the "sure thing principle". It says that "when comparing two decisions, it is not necessary to consider states of nature in which these decisions yield the same outcome." Another key axiom is the idea that the likelihood ranking of events does not depend on the consequences (such as receiving a car rather than receiving 100 euros). Savage's theory resembles von Neumann and Morgenstern expected utility under risk, meaning that "decision under uncertainty can in some sense be reduced to decision under risk, with one important caveat: beliefs are here a purely subjective construct."

Etner et al then discuss models generalizing Savage expected utility using non-Bayesian decision criteria. The first model is the Wald maxmin criterion, which evaluates decisions by looking exclusively at the worst possible consequence. It was generalized by Arrow and Hurwicz in 1972, who consider a convex combination of both the worst and the best outcomes (this is called the Arrow and Hurwicz alpha maxmin model). The weight of the worst outcome, denoted alpha, measures the decision maker's pessimism. The Arrow and Hurwicz model has been shown to be the only criterion susceptible to model choice under complete ignorance, i.e., the decision maker has no means to assess whether one event is more likely than another. Etner et al then present two "first-generation" models that use capacities to represent non-additive beliefs in the face of uncertain situations: the Choquet expected utility and the cumulative prospect theory. In the case of Choquet expected utility, beliefs are characterized not by a subjective probability but by a capacity, that is, a non-necessarily additive, increasing set function. Actions are then evaluated using Choquet integrals rather than standard Lebesgue integrals due to the non-additivity of the capacity. For decisions with a finite set of outcomes, "a decision maker evaluates a decision by considering first the lowest outcome and then adding to this lowest outcome the successive possible increments, weighted by his personal estimation of the occurrence of these increments." The cumulative prospect theory developed by Kahneman and Tversky in 1979 is related to the Choquet expected utility model but has a reference point and an asymmetry in the treatment of gains and losses. It introduces two different capacities, one for gains and one for losses.

The next generation of models of decision making under uncertainty "rests on the idea that when information is sccarce, it is too demanding to ask for precise subjectiee beliefs (a probability distribution) but maybe asking only for 'imprecise' subjective beliefs (i.e., a set of probability distributions) is more appropriate." This is known as having multiple priors. It is still possible to compute expected utilities, but now we have one value per prior. One possibility to compare two decisions is to say that an act f is preferred to an act g if all the expected utilities of f with respect to the distributions in the set of priors are higher than those of g. As a result, not all acts can be compared (f can be better than g for some priors and g can be better than f for others.) This is an example of incomplete preferences. It is possible to evaluate the multiple priors by considering their worst case, which gives rise to the maxmin expected utility of Gilboa and Schmeidler. Under an assumption of uncertainty aversion, "the Choquet expected utility model is a particular case of the maxmin expected utility model" and the set of priors over which the decision maker takes the minimum is the core of a convex capacity. The maxmin expected utility can also be generalized using a confidence function, using a threshold level under which priors are not taken into account in the evaluation.

A natural follow-up question is how to determine the set of priors in the maxmin expected utility model. That, and the rest of the paper, will be the subject of my next post!

One of the most useful features of the paper is a toy insurance example that runs throughout the paper to illustrate the various concepts.

1. An individual with initial wealth w is facing a risk of loss d. The state space is: {Loss, No Loss}. An act specifies what amount of money the decision maker has in each state of nature. The act of not buying insurance would be represented by f = (w-d, w). Buying full coverage at premium pi would yield g = (w - pi, w - pi). Buying partial coverage at premium pi' would lead to h = (w-d + I - pi'), w - pi') with I the indemnity paid in case of damage.
2. A decision maker following Savage's axioms would have a probability of loss p and a utility function u and would compute p*u(w-d)+(1-p)*u(w) for act f, u(w-pi) for act g and p*(w-d + I - pi')+(1-p)*u(w - pi') for act h.
3. If the DM's preferences are represented by the Choquet expected utility model, the decisions will be evaluated as: for act f, u(w-d)+v(NoLoss)*[u(w)-u(w-d)], for act g, u(w-pi) and for act h, u(w-d+I-pi')+v(NoLoss)*[u(w-pi')-u(w-d+I-pi')]
4. Incomplete preferences can be illustrated assuming that p (the probability of loss) is either 1/3 or 1/2 and taking w=3/2, d=1/2 and u(x)=x. If 4/3>3/2-pi >5/4, "it is sometimes better to get full insurance and sometimes better not to have any insurance" depending on the prior used.
5. If the individual's subjective set of beliefs is such that p is in the range [p',p''] and he decides using a maxmin expected utility model, then he evaluates f as p''u(w-d) + (1-p'')u(w), g as u(w-pi) and h as p''u(w-d+I-pi')+(1-p'')u(w-pi'). Note that the individual "evaluates acts only according to the worst possible prior in the set, that is, the highest loss probability p''."
6. Starting with the sixth part of the example the math starts being too cumbersome to retype so you will have to look up the rest in the paper itself!

 Today's post is on Procurement with Unforeseen Contingencies by Fabian Herweg and Klaus Schmidt, which was published last year in 灯蓝专业版免费 安卓. This is about buyers who want to procure complex goods, whose designs they have made themselves. In price-only auctions, the seller/contractor who will make the product based on the design has no incentive to share his expertise, which would minimize design adjustment costs. The authors propose a mechanism that both "allocates the contract to the seller with the lowest cost and that induces all potential sellers to reveal any information that they may have about possible design flaws early (i.e., before the contract is assigned)." The authors' approach takes into account that, although the buyer knows the potential design may be flawed, he has "no idea what possible flaws look like and what their payoff implications are." 

The authors explain their mechanism as follows: sellers have to point out any design few they discovered to an independent arbitrator, who verifies the flaws, estimates their payoff consequences and assigns payments to the sellers as follows: "if a seller is the only one who reported a flaw, he will get a reward that is equal to the additional profit he cud have made if he had kept the information to himself, won the contract, and renegotiated with the buyer; otherwise, he gets nothing." Then "the revealed flaws are fixed and the improved design is allocated to the seller with the lowest cost in a sample Vickrey auction."  

蓝灯vp:1 天前 · 蓝灯lantern破解版vip无限流量及各版本下载 | 路由器博客 [图文] 网站首页 >游戏助手>蓝灯VIP破解版v3.6.6破解内购专业版 蓝灯VIP破解版v3.6.6破解内购专业版 蓝灯的Android版本怎么用_ 点中间的按钮.选择确定.翻墙成功的话会在通知栏有一个小钥匙的标志.

Today's post is on a survey of multiple-objective sequential decision-making that appeared in the Journal of Artificial intelligence Research in 2013. The authors "identify three distinct scenarios in which converting such a problem to a single-objective one is impossible, infeasible or undesirable." They propose a classification of multi-objective methods that determines the nature of a optimal solution, "which can be a single policy, a convex hull or a Pareto front." 

They start by formalizing the problem as a multi-objective Markov Decision Process (MOMDP) and then motivate the multi-objective perspective on decision-making (rather than combining the multiple objectives into one function via scalarization). The three scenarios for which this makes sense are:

1. The unknown weights scenario, meaning that the weights are unknown when the planning happens. (For instance, the arises if you are trying to balance commuting time and pollution costs in a public transport system, and the price to offset pollution costs, traded on an open market, varies every day.) 
2. The decision support scenario, meaning that you cannot easily turn all the objectives into a common currency such as dollars. 
3. The known weights scenario, meaning that we could scalarize the muti-objective problem into a single-objective problem but this may not be desirable if the scalarization function is nonlinear, because the optimal policy could then be non-stationary or stochastic, which cannot happen in single-objective, additive, infinite-horizon MDPs. As a result, standard methods are not applicable and the MDP can be difficult to solve.  

They present "a novel taxonomy that organizes multi-objective problems in terms of their underlying assumptions and the nature of the resulting solutions" in Table 1 p. "75" of the paper I linked above (page 1 is numbered p.67, so don't let those numbers scare you.) They discuss the set of undominated policies for the MOMDP and the concept of coverage set, i.e., a subset of undominated policies for which, for each weight vector, at least one policy in the subset is optimal.

The authors consider two types of scalarization functions: linear (which are more intuitive) and monotonically increasing (which can express more accurately the user's preferences). For linear scalarization functions they define the convex hull of policies and for monotonically increasing scalarization functions they define the concepts of Pareto front and Pareto coverage set. It is also important to distinguish between deterministic and stochastic policies.

In single-objective MDPs, the manager can restrict his search to deterministic stationary policies. This is also true of MOMDPs when the scalarization function is linear. The authors show that "in the multiple-policy setting when only deterministic policies are allowed and [the scalarization function] is nonlinear, non-stationary policies may be better than the best stationary ones." When stochastic non-stationary policies are allowed, the manager can use stochastic stationary policies instead of deterministic non-stationary ones. Hence, the authors explain, the manager "can employ a mixture policy and selects the i-th policy for this set with probability p_i."  

Finally, they survey MOMDP planning methods, i.e., "computing an optimal policy or the coverage set of undominated policies given a complete model of the MOMDP", and MOMDP learning methods, when a complete model of the MOMDP is not available. In that case, multi-objective reinforcement learning (MORL) can be used. The majority of approaches in that field is on single policies, learnt through standard temporal-difference (TD) methods such as Q-learning. Linear scalarization is often considered but is, again, not always appropriate.

The four main classes of single-policy MORL with non-linear secularization are:

a. TD methods without modification (heuristics not guaranteed to converge),

b. modified TD algorithm or state representation to avoid non-additive returns,

c. TD methods to learn multiple polices using linear scalarization with different values for the weight vector,

d. policy-search methods.

In the case of multiple-policy learning with linear scalarization, running TD multiple times wth different weight vectors is inefficient, and more sophisticated approaches to learn multiple policies in parallel through online learning or to implement offline a multi-objective extension of the Fitted Q-Iteration algorithm.

The last sections survey the applications that have used multi-objective methods for planning and learning and discuss future work. Overall, a good read, highly informative.