Tools for exporting chat related portions of Langchain runs.
from nbdev.showdoc import show_doc

When instantiating ChatRecord with the class methods ChatRecord.from_run or ChatRecord.from_run_id, we automatically query the parent run of the LangChain trace in LangSmith to get metadata like feedback. Additionally, if you instantiate ChatRecord with a root run or a run that is not a ChatOpenAI run type, ChatRecord will attempt to find the last ChatOpenAI in your chain and store the id in ChatRecord.child_run_id. The data for this child run (inputs, outputs, functions) is stored in ChatRecord.child_run and is of type RunData.



 ChatRecord.from_run (run:langsmith.schemas.Run)

Collect information About A Run into a ChatRecord.

Type Details
run Run the run object to parse.
client = Client()
_root_run = client.read_run('fbfd220a-c731-46a2-87b3-e64a477824f5')
_root_result = ChatRecord.from_run(_root_run)



 ChatRecord.from_run_id (run_id:str)

Collect information About A Run into a ChatRecord.

Type Details
run_id str the run id to fetch and parse.
_child_run_id = str(_root_run.child_run_ids[-1])
_child_result = ChatRecord.from_run_id(_child_run_id)


# test that child and root runs are related
test_eq(_root_result.flat_output, _child_result.flat_output)
test_eq(_root_result.parent_run_id, _child_result.parent_run_id)

# Test case without feedback
_parent_run_no_feedback = client.read_run('87900cfc-0322-48fb-b009-33d226d73597')
_no_feedback = ChatRecord.from_run(_parent_run_no_feedback)
test_eq(, [])

# Test case with feedback

#  ... starting with a child run
_child_w_feedback = client.read_run('f8717b0e-fb90-45cd-be00-9b4614965a2e')
_feedback = ChatRecord.from_run(_child_w_feedback).feedback
assert _feedback[0]['key'] == 'empty response'

# #  ... starting with a parent run
_parent_w_feedback = client.read_run(_child_w_feedback.parent_run_id)
_feedback2 = ChatRecord.from_run(_parent_w_feedback).feedback
test_eq(_feedback[0]['comment'],  _feedback2[0]['comment'])

ChatRecordSet, a list of ChatRecord



 ChatRecordSet.from_runs (runs:List[langsmith.schemas.Run])

Load ChatRecordSet from runs.

We can create a ChatRecordSet directly from a list of runs:

# from langfree.runs import get_runs_by_commit
_runs = get_runs_by_commit(commit_id='028e4aa4', limit=10)
llmdata = ChatRecordSet.from_runs(_runs)
Fetching runs with this filter: and(eq(status, "success"), has(tags, "commit:028e4aa4"))

There is a special shortcut to get runs by a commit tag which uses get_runs_by_commit for you:



 ChatRecordSet.from_commit (commit_id:str, limit:int=None)

Create a ChatRecordSet from a commit id

llmdata2 = ChatRecordSet.from_commit('028e4aa4', limit=10)
assert llmdata[0].child_run_id == llmdata2[0].child_run_id
Fetching runs with this filter: and(eq(status, "success"), has(tags, "commit:028e4aa4"))

Finally, you can also construct a ChatRecordSet from a list of run ids:



 ChatRecordSet.from_run_ids (runs:List[str])

Load ChatRecordSet from run ids.

_run_ids = ['ba3c0a47-0803-4b0f-8a2f-380722edc2bf',

llmdata3 = ChatRecordSet.from_run_ids(_run_ids)
assert len(llmdata3) == len(_run_ids)
assert llmdata[0].child_run_id == _run_ids[0]

Convert ChatRecordSet to a Pandas Dataframe

You can do this with to_pandas()

_df = llmdata.to_pandas()
child_run_id child_run child_url parent_run_id parent_url total_tokens prompt_tokens completion_tokens feedback feedback_keys tags start_dt function_defs param_model_name param_n param_top_p param_temp param_presence_penalty param_freq_penalty
0 ba3c0a47-0803-4b0f-8a2f-380722edc2bf inputs=[{'role': 'system', 'content': 'You are... 7074af93-1821-4325-9d45-0f2e81eca0fe 0 0 0 [] [] [commit:028e4aa4, branch:testing, test, room:6... 09/05/2023 [{'name': 'contact-finder', 'parameters': {'ty... gpt-3.5-turbo-0613 1 1 0 0 0

Save Data'_data/llm_data.pkl')

Load Data

_loaded = ChatRecordSet.load('_data/llm_data.pkl')
assert llmdata.records[0].child_run_id == _loaded.records[0].child_run_id