• Document: Boosting for Transfer Learning
  • Size: 186.45 KB
  • Uploaded: 2019-07-15 03:21:52
  • Status: Successfully converted

Some snippets from your converted document:

Boosting for Transfer Learning Wenyuan Dai dwyak@apex.sjtu.edu.cn Department of Computer Science and Engineering, Shanghai Jiao Tong University, China Qiang Yang qyang@cse.ust.hk Deptarment of Computer Science, Hong Kong University of Science and Technology, Hong Kong Gui-Rong Xue grxue@apex.sjtu.edu.cn Yong Yu yyu@apex.sjtu.edu.cn Department of Computer Science and Engineering, Shanghai Jiao Tong University, China Abstract 1. Introduction Traditional machine learning makes a ba- A fundamental assumption in classification learning is sic assumption: the training and test data that the data distributions of training and test sets should be under the same distribution. should be identical. When the assumption does not However, in many cases, this identical- hold, traditional classification methods might perform distribution assumption does not hold. The worse. However, in practice, this assumption may not assumption might be violated when a task always hold. For example, in Web mining, the Web from one new domain comes, while there data used in training a Web-page classification model are only labeled data from a similar old can be easily out-dated when applied to the Web some- domain. Labeling the new data can be time later, because the topics on the web change fre- costly and it would also be a waste to quently. Often, new data are expensive to label and throw away all the old data. In this pa- thus their quantities are limited due to cost issues. per, we present a novel transfer learning How to accurately classify the new test data by mak- framework called TrAdaBoost, which extends ing the maximum use of the old data becomes a critical boosting-based learning algorithms (Freund problem. & Schapire, 1997). TrAdaBoost allows users Although the training data are more or less out-dated, to utilize a small amount of newly labeled there are certain parts of the data that can still be data to leverage the old data to construct a reused. That is, knowledge learned from this part of high-quality classification model for the new the data can still be of use in training a classifier for data. We show that this method can allow the new data. To find out which those data are, we us to learn an accurate model using only a employ a small amount of labeled new data, called tiny amount of new data and a large amount same-distribution training data, to help vote on the of old data, even when the new data are not usefulness of each of the old data instance. Because sufficient to train a model alone. We show some of the old training data might be out-dated and that TrAdaBoost allows knowledge to be ef- be under a different distribution from the new data, we fectively transferred from the old data to the call them diff-distribution training data. Our goal is new. The effectiveness of our algorithm is an- to learn a high-quality classification model using both alyzed theoretically and empirically to show the same-distribution and diff-distribution data. This that our iterative algorithm can converge well learning process corresponds to transferring knowledge to an accurate model. learned from the old data to new situations – an in- stance of transfer learning. Appearing in Proceedings of the 24 th International Confer- In this paper, we try to develop a general framework ence on Machine Learning, Corvallis, OR, 2007. Copyright for transfer learning based on (Freund & Schapire, 2007 by the author(s)/owner(s). 1997), and analyze the correctness of the general model Boosting for Transfer Learning using the Probability Approximately Correct (PAC) Our problem setting can also be considered as learning theory. Our key idea is to use boosting to filter out with auxiliary data, where the labeled diff-distribution the diff-distribution training data that are very differ- data are treated as the auxiliary data. In previous ent from the same-distribution data by automatically works, Wu and Dietterich (2004) proposed an image adju

Recently converted files (publicly available):