Dev Stack Source

Answer : You can't: DataFrame columns are Series , by definition. That said, if you make the dtype (the type of all the elements) datetime-like, then you can access the quantities you want via the .dt accessor (docs): >>> df["TimeReviewed"] = pd.to_datetime(df["TimeReviewed"]) >>> df["TimeReviewed"] 205 76032930 2015-01-24 00:05:27.513000 232 76032930 2015-01-24 00:06:46.703000 233 76032930 2015-01-24 00:06:56.707000 413 76032930 2015-01-24 00:14:24.957000 565 76032930 2015-01-24 00:23:07.220000 Name: TimeReviewed, dtype: datetime64[ns] >>> df["TimeReviewed"].dt <pandas.tseries.common.DatetimeProperties object at 0xb10da60c> >>> df["TimeReviewed"].dt.year 205 76032930 2015 232 76032930 2015 233 76032930 2015 413 76032930 2015 565 76032930 2015 dtype: int64 >>> df["TimeReviewed"].dt.month 205 76032930 1 232 76032930 1 233 7...

Answer : The to_dict() method sets the column names as dictionary keys so you'll need to reshape your DataFrame slightly. Setting the 'ID' column as the index and then transposing the DataFrame is one way to achieve this. to_dict() also accepts an 'orient' argument which you'll need in order to output a list of values for each column. Otherwise, a dictionary of the form {index: value} will be returned for each column. These steps can be done with the following line: >>> df.set_index('ID').T.to_dict('list') {'p': [1, 3, 2], 'q': [4, 3, 2], 'r': [4, 0, 9]} In case a different dictionary format is needed, here are examples of the possible orient arguments. Consider the following simple DataFrame: >>> df = pd.DataFrame({'a': ['red', 'yellow', 'blue'], 'b': [0.5, 0.25, 0.125]}) >>> df a b 0 red 0.500 1 yellow 0.250 2 blue 0.125 The...

Answer : Use groupby and agg , and aggregate only unique values by calling Series.unique : df.astype(str).groupby('prop1').agg(lambda x: ','.join(x.unique())) prop2 prop3 prop4 prop1 K20 12,1,66 travis,leo 10.0,4.0 L30 3,54,11,10 bob,john 11.2,10.0 df.astype(str).groupby('prop1', sort=False).agg(lambda x: ','.join(x.unique())) prop2 prop3 prop4 prop1 L30 3,54,11,10 bob,john 11.2,10.0 K20 12,1,66 travis,leo 10.0,4.0 If handling NaNs is important, call fillna in advance: import re df.fillna('').astype(str).groupby('prop1').agg( lambda x: re.sub(',+', ',', ','.join(x.unique())) ) prop2 prop3 prop4 prop1 K20 12,1,66 travis,leo 10.0,4.0 L30 3,54,11,10 bob,john 11.2,10.0

Answer : I don't think your question makes sense -- your outermost Map , I only see you are trying to stuff values into it -- you need to have key / value pairs in your outermost Map . That being said: val peopleArray = df.collect.map(r => Map(df.columns.zip(r.toSeq):_*)) Will give you: Array( Map("age" -> null, "name" -> "Michael"), Map("age" -> 30, "name" -> "Andy"), Map("age" -> 19, "name" -> "Justin") ) At that point you could do: val people = Map(peopleArray.map(p => (p.getOrElse("name", null), p)):_*) Which would give you: Map( ("Michael" -> Map("age" -> null, "name" -> "Michael")), ("Andy" -> Map("age" -> 30, "name" -> "Andy")), ("Justin" -> Map("age" -> 19, "name" -> "Justin")) ) I'm guess...

Answer : Pandas concat takes a list of dataframes. If you can generate a list of dataframes with your looping function, once you are finished you can concatenate the list together: data_day_list = [] for i, day in enumerate(list_day): data_day = df[df.day==day] data_day_list.append(data_day) final_data_day = pd.concat(data_day_list) Exhausting a generator is more elegant (if not more efficient) than appending to a list. For example: def yielder(df, list_day): for i, day in enumerate(list_day): yield df[df['day'] == day] final_data_day = pd.concat(list(yielder(df, list_day)) Appending or concatenating pd.DataFrame s is slow. You can use a list in the interim and then create the final pd.DataFrame at the end with pd.DataFrame.from_records() e.g.: interim_list = [] for i,(k,g) in enumerate(df.groupby(['[*name of your date column here*'])): if i % 1000 == 0 and i != 0: print('iteration: {}'.format(i)) # just tells you where you are i...

Answer : Use the names function: whatyouwant <- as.character(dd$name) names(whatyouwant) <- dd$crit as.character is necessary, because data.frame and read.table turn characters into factors with default settings. If you want a one-liner: whatyouwant <- setNames(as.character(dd$name), dd$crit) You can also use deframe(x) from the tibble package for this. tibble::deframe() It converts the first column to names and second column to values. You can make a vector from dd$name , and add names using names() , but you can do it all in one step with structure() : whatiwant <- structure(as.character(dd$name), names = as.character(dd$crit))