- Joined
- 8/5/14
- Messages
- 312
- Points
- 303
I am currently in my fourth year (undergrad) studying Applied Mathematics with specialization in mechanical controls. I have to decide on my fourth year project topic very soon. I am hoping to make it related to some area of finance. I am in quandary between 3 areas of research:
Project 1. Learning Algorithms in Team and Game Problems (with focus on static game theory)
Project 2. Control along trajectories (ie. linearizing an LTV system along a non-stationary trajectory)
Project 3. Stochastic Network Control Systems
I have provided more detailed descriptions below; however, the topics are very open-ended and I could tailor the scope of the project. I have the most general background knowledge of project 2 and it is related to linearization of LTI control system along a 'non-stationary' trajectory (apparently this is very difficult according to the prof.). Obviously, project 2 has nothing to do with finance (but it's interesting to me). Besides a little bit of research I have done over the past week, I know very little about Project 2 and Project 3. However, this is not an issue as my professor/supervisor will provide all background information during office hours or lectures.
Which project should I choose? Which project seems most interesting? Any suggestions would be great. I can provide more information. Thanks!
-----------------------------------------------------------------------------------------------------------------------------------
Background info:
Project 1. Learning Algorithms in Team and Game Problems (with focus on static game theory)
In many engineering applications, one has to design optimal decision and control policies in an environment which is not completely known to a decision maker. For such problems, a decision maker typically applies experimentation to obtain new knowledge about the system and and opti- mizes his/her decision based on the past and the new information.
If there is only one decision maker acting in an environment that is stationary (for example an investor which takes part in an economy with invariant characteristics or a gambler which picks a slot machine to play) then one can apply certain algorithms which eventually converge to optimal policies.
On the other hand, when there are multiple decision makers present in a system, since each of the multiple decision makers take part in learning, the environment seen by a single decision maker is not stationary. For instance, if two soccer teams play with each other for the first time, they may not know their play style (attacking-defensive etc.). Learning for such contexts is challenging, but many physical systems admit such setups.
The goal in this project is to design and implement learning strategies to compute optimal solutions for team problems, or policies which converge to equilibrium solutions. In particular, a variation of a popular method known as Q-learning will be applied to team and game problems, and the limitations and use of this algorithm will be identified.
There will be research and implementation components for this project. Background information and further references will be provided.
Project 2. Control along trajectories
In control theory one very often does control about a prescribed reference trajectory, with the typical objective to make the trajectory stable in some sense by the use of control. Up to this point in your young lives, the reference trajectory has typically been an equilibrium point, and you have learnt a variety of methods for stabilisation using linearisation. In this project, the goal is to stabilise a trajectory that is not stationary. You will have to (very quickly) decide upon a specific system and a specific reference trajectory of that system about which to linearise. Then you will have to learn about techniques for stabilising about a non-stationary trajectory, and implement these in software.
The objectives of the project are the following.
Project 3. Stochastic Networked Control Systems:
"Control systems are increasingly becoming decentralized and networked: Decentralized because the information available at the controllers might be different and imperfect, and networked because there may be communication channels between various components of a control system (such as sensors, controllers and actuators). In this project, we will try to understand some salient issues in such decentralized and networked systems. The application areas for such problems span many fields in engineering, economics and applied mathematics.
As an example, consider an n−dimensional discrete-time system: xt+1 =f(xt,u1t,u2t,wt),
with observations at two different sensors
y t1 = g 1 ( x t , v t1 ) y t2 = g 2 ( x t , v t2 )
Here, v1, v2, w are disturbance/noise variables, ui, i = 1, 2 are the control actions. Sensor i has access to yi variables and needs to transmit this information to a controller. A controller i is required to use only its local observations while generating control actions.
The goal will be to (i) stabilize or (ii) optimize such a system subject to such information constraints:
The goal is to understand the value of limited information for control of such problems in a networked environment. There will be research and implementation components for this project. Background information and further references will be provided. "
-----------------------------------------------------------------------------------------------------------------------------------
Project 1. Learning Algorithms in Team and Game Problems (with focus on static game theory)
Project 2. Control along trajectories (ie. linearizing an LTV system along a non-stationary trajectory)
Project 3. Stochastic Network Control Systems
I have provided more detailed descriptions below; however, the topics are very open-ended and I could tailor the scope of the project. I have the most general background knowledge of project 2 and it is related to linearization of LTI control system along a 'non-stationary' trajectory (apparently this is very difficult according to the prof.). Obviously, project 2 has nothing to do with finance (but it's interesting to me). Besides a little bit of research I have done over the past week, I know very little about Project 2 and Project 3. However, this is not an issue as my professor/supervisor will provide all background information during office hours or lectures.
Which project should I choose? Which project seems most interesting? Any suggestions would be great. I can provide more information. Thanks!
-----------------------------------------------------------------------------------------------------------------------------------
Background info:
Project 1. Learning Algorithms in Team and Game Problems (with focus on static game theory)
In many engineering applications, one has to design optimal decision and control policies in an environment which is not completely known to a decision maker. For such problems, a decision maker typically applies experimentation to obtain new knowledge about the system and and opti- mizes his/her decision based on the past and the new information.
If there is only one decision maker acting in an environment that is stationary (for example an investor which takes part in an economy with invariant characteristics or a gambler which picks a slot machine to play) then one can apply certain algorithms which eventually converge to optimal policies.
On the other hand, when there are multiple decision makers present in a system, since each of the multiple decision makers take part in learning, the environment seen by a single decision maker is not stationary. For instance, if two soccer teams play with each other for the first time, they may not know their play style (attacking-defensive etc.). Learning for such contexts is challenging, but many physical systems admit such setups.
The goal in this project is to design and implement learning strategies to compute optimal solutions for team problems, or policies which converge to equilibrium solutions. In particular, a variation of a popular method known as Q-learning will be applied to team and game problems, and the limitations and use of this algorithm will be identified.
There will be research and implementation components for this project. Background information and further references will be provided.
Project 2. Control along trajectories
In control theory one very often does control about a prescribed reference trajectory, with the typical objective to make the trajectory stable in some sense by the use of control. Up to this point in your young lives, the reference trajectory has typically been an equilibrium point, and you have learnt a variety of methods for stabilisation using linearisation. In this project, the goal is to stabilise a trajectory that is not stationary. You will have to (very quickly) decide upon a specific system and a specific reference trajectory of that system about which to linearise. Then you will have to learn about techniques for stabilising about a non-stationary trajectory, and implement these in software.
The objectives of the project are the following.
- (a) Determine a system and reference trajectory that is tractable without being trivial.
- (b) Learn about suitable control methodologies.
- (c) Understand and quantify the desired properties of your control law.
- (d) Implement various methodologies in software.
- (e) Quantitatively compare the various methodologies.
- (f) Understand the broad engineering context for the project, and the sort of research that will go into it.
- (g) Work effectively and professionally, as a group and individually.
- (h) Give effective presentations.
- (i) Write an excellent report.
Project 3. Stochastic Networked Control Systems:
"Control systems are increasingly becoming decentralized and networked: Decentralized because the information available at the controllers might be different and imperfect, and networked because there may be communication channels between various components of a control system (such as sensors, controllers and actuators). In this project, we will try to understand some salient issues in such decentralized and networked systems. The application areas for such problems span many fields in engineering, economics and applied mathematics.
As an example, consider an n−dimensional discrete-time system: xt+1 =f(xt,u1t,u2t,wt),
with observations at two different sensors
y t1 = g 1 ( x t , v t1 ) y t2 = g 2 ( x t , v t2 )
Here, v1, v2, w are disturbance/noise variables, ui, i = 1, 2 are the control actions. Sensor i has access to yi variables and needs to transmit this information to a controller. A controller i is required to use only its local observations while generating control actions.
The goal will be to (i) stabilize or (ii) optimize such a system subject to such information constraints:
The goal is to understand the value of limited information for control of such problems in a networked environment. There will be research and implementation components for this project. Background information and further references will be provided. "
-----------------------------------------------------------------------------------------------------------------------------------
