A ScheduleDAG for scheduling lists of MachineInstr. More...

#include "llvm/CodeGen/ScheduleDAGInstrs.h"

Inheritance diagram for llvm::ScheduleDAGInstrs:

[legend]

Collaboration diagram for llvm::ScheduleDAGInstrs:

[legend]

Classes
class	Value2SUsMap

Public Types
using	SUList = std::list< SUnit * >
	A list of SUnits, used in Value2SUsMap, during DAG construction. More...

Public Member Functions
	ScheduleDAGInstrs (MachineFunction &mf, const MachineLoopInfo *mli, bool RemoveKillFlags=false)

	~ScheduleDAGInstrs () override=default

const TargetSchedModel *	getSchedModel () const
	Gets the machine model for instruction scheduling. More...

const MCSchedClassDesc *	getSchedClass (SUnit *SU) const
	Resolves and cache a resolved scheduling class for an SUnit. More...

MachineBasicBlock::iterator	begin () const
	Returns an iterator to the top of the current scheduling region. More...

MachineBasicBlock::iterator	end () const
	Returns an iterator to the bottom of the current scheduling region. More...

SUnit *	newSUnit (MachineInstr *MI)
	Creates a new SUnit and return a ptr to it. More...

SUnit *	getSUnit (MachineInstr *MI) const
	Returns an existing SUnit for this MI, or nullptr. More...

virtual bool	doMBBSchedRegionsTopDown () const
	If this method returns true, handling of the scheduling regions themselves (in case of a scheduling boundary in MBB) will be done beginning with the topmost region of MBB. More...

virtual void	startBlock (MachineBasicBlock *BB)
	Prepares to perform scheduling in the given block. More...

virtual void	finishBlock ()
	Cleans up after scheduling in the given block. More...

virtual void	enterRegion (MachineBasicBlock *bb, MachineBasicBlock::iterator begin, MachineBasicBlock::iterator end, unsigned regioninstrs)
	Initialize the DAG and common scheduler state for a new scheduling region. More...

virtual void	exitRegion ()
	Called when the scheduler has finished scheduling the current region. More...

void	buildSchedGraph (AliasAnalysis AA, RegPressureTracker RPTracker=nullptr, PressureDiffs PDiffs=nullptr, LiveIntervals LIS=nullptr, bool TrackLaneMasks=false)
	Builds SUnits for the current region. More...

void	addSchedBarrierDeps ()
	Adds dependencies from instructions in the current list of instructions being scheduled to scheduling barrier. More...

virtual void	schedule ()=0
	Orders nodes according to selected style. More...

virtual void	finalizeSchedule ()
	Allow targets to perform final scheduling actions at the level of the whole MachineFunction. More...

void	dumpNode (const SUnit &SU) const override

void	dump () const override

std::string	getGraphNodeLabel (const SUnit *SU) const override
	Returns a label for a DAG node that points to an instruction. More...

std::string	getDAGName () const override
	Returns a label for the region of code covered by the DAG. More...

void	fixupKills (MachineBasicBlock &MBB)
	Fixes register kill flags that scheduling has made invalid. More...

Public Member Functions inherited from llvm::ScheduleDAG
	ScheduleDAG (MachineFunction &mf)

virtual	~ScheduleDAG ()

void	clearDAG ()
	Clears the DAG state (between regions). More...

const MCInstrDesc *	getInstrDesc (const SUnit *SU) const
	Returns the MCInstrDesc of this SUnit. More...

virtual void	viewGraph (const Twine &Name, const Twine &Title)
	Pops up a GraphViz/gv window with the ScheduleDAG rendered using 'dot'. More...

virtual void	viewGraph ()
	Out-of-line implementation with no arguments is handy for gdb. More...

void	dumpNodeName (const SUnit &SU) const

virtual void	addCustomGraphFeatures (GraphWriter< ScheduleDAG *> &) const
	Adds custom features for a visualization of the ScheduleDAG. More...

unsigned	VerifyScheduledDAG (bool isBottomUp)
	Verifies that all SUnits were scheduled and that their state is consistent. More...

Protected Types
using	DbgValueVector = std::vector< std::pair< MachineInstr , MachineInstr > >

Protected Member Functions
void	reduceHugeMemNodeMaps (Value2SUsMap &stores, Value2SUsMap &loads, unsigned N)
	Reduces maps in FIFO order, by N SUs. More...

void	addChainDependency (SUnit SUa, SUnit SUb, unsigned Latency=0)
	Adds a chain edge between SUa and SUb, but only if both AliasAnalysis and Target fail to deny the dependency. More...

void	addChainDependencies (SUnit *SU, SUList &SUs, unsigned Latency)
	Adds dependencies as needed from all SUs in list to SU. More...

void	addChainDependencies (SUnit *SU, Value2SUsMap &Val2SUsMap)
	Adds dependencies as needed from all SUs in map, to SU. More...

void	addChainDependencies (SUnit *SU, Value2SUsMap &Val2SUsMap, ValueType V)
	Adds dependencies as needed to SU, from all SUs mapped to V. More...

void	addBarrierChain (Value2SUsMap &map)
	Adds barrier chain edges from all SUs in map, and then clear the map. More...

void	insertBarrierChain (Value2SUsMap &map)
	Inserts a barrier chain in a huge region, far below current SU. More...

void	initSUnits ()
	Creates an SUnit for each real instruction, numbered in top-down topological order. More...

void	addPhysRegDataDeps (SUnit *SU, unsigned OperIdx)
	MO is an operand of SU's instruction that defines a physical register. More...

void	addPhysRegDeps (SUnit *SU, unsigned OperIdx)
	Adds register dependencies (data, anti, and output) from this SUnit to following instructions in the same scheduling region that depend the physical register referenced at OperIdx. More...

void	addVRegDefDeps (SUnit *SU, unsigned OperIdx)
	Adds register output and data dependencies from this SUnit to instructions that occur later in the same scheduling region if they read from or write to the virtual register defined at OperIdx. More...

void	addVRegUseDeps (SUnit *SU, unsigned OperIdx)
	Adds a register data dependency if the instruction that defines the virtual register used at OperIdx is mapped to an SUnit. More...

void	startBlockForKills (MachineBasicBlock *BB)
	Initializes register live-range state for updating kills. More...

void	toggleKillFlag (MachineInstr &MI, MachineOperand &MO)
	Toggles a register operand kill flag. More...

LaneBitmask	getLaneMaskForMO (const MachineOperand &MO) const
	Returns a mask for which lanes get read/written by the given (register) machine operand. More...

Protected Member Functions inherited from llvm::ScheduleDAG
void	dumpNodeAll (const SUnit &SU) const

Protected Attributes
const MachineLoopInfo *	MLI

const MachineFrameInfo &	MFI

TargetSchedModel	SchedModel
	TargetSchedModel provides an interface to the machine model. More...

bool	RemoveKillFlags
	True if the DAG builder should remove kill flags (in preparation for rescheduling). More...

bool	CanHandleTerminators = false
	The standard DAG builder does not normally include terminators as DAG nodes because it does not create the necessary dependencies to prevent reordering. More...

bool	TrackLaneMasks = false
	Whether lane masks should get tracked. More...

MachineBasicBlock *	BB
	The block in which to insert instructions. More...

MachineBasicBlock::iterator	RegionBegin
	The beginning of the range to be scheduled. More...

MachineBasicBlock::iterator	RegionEnd
	The end of the range to be scheduled. More...

unsigned	NumRegionInstrs
	Instructions in this region (distance(RegionBegin, RegionEnd)). More...

DenseMap< MachineInstr , SUnit >	MISUnitMap
	After calling BuildSchedGraph, each machine instruction in the current scheduling region is mapped to an SUnit. More...

Reg2SUnitsMap	Defs
	Defs, Uses - Remember where defs and uses of each register are as we iterate upward through the instructions. More...

Reg2SUnitsMap	Uses

VReg2SUnitMultiMap	CurrentVRegDefs
	Tracks the last instruction(s) in this region defining each virtual register. More...

VReg2SUnitOperIdxMultiMap	CurrentVRegUses
	Tracks the last instructions in this region using each virtual register. More...

AliasAnalysis *	AAForDep = nullptr

SUnit *	BarrierChain = nullptr
	Remember a generic side-effecting instruction as we proceed. More...

UndefValue *	UnknownValue
	For an unanalyzable memory access, this Value is used in maps. More...

DbgValueVector	DbgValues
	Remember instruction that precedes DBG_VALUE. More...

MachineInstr *	FirstDbgValue = nullptr

LivePhysRegs	LiveRegs
	Set of live physical registers for updating kill flags. More...

Additional Inherited Members
Public Attributes inherited from llvm::ScheduleDAG
const LLVMTargetMachine &	TM
	Target processor. More...

const TargetInstrInfo *	TII
	Target instruction information. More...

const TargetRegisterInfo *	TRI
	Target processor register info. More...

MachineFunction &	MF
	Machine function. More...

MachineRegisterInfo &	MRI
	Virtual/real register map. More...

std::vector< SUnit >	SUnits
	The scheduling units. More...

SUnit	EntrySU
	Special node for the region entry. More...

SUnit	ExitSU
	Special node for the region exit. More...

bool	StressSched

Detailed Description

A ScheduleDAG for scheduling lists of MachineInstr.

Definition at line 119 of file ScheduleDAGInstrs.h.

Member Typedef Documentation

◆ DbgValueVector

using llvm::ScheduleDAGInstrs::DbgValueVector = std::vector<std::pair<MachineInstr *, MachineInstr *> >

protected

Definition at line 239 of file ScheduleDAGInstrs.h.

◆ SUList

using llvm::ScheduleDAGInstrs::SUList = std::list<SUnit *>

A list of SUnits, used in Value2SUsMap, during DAG construction.

Note: to gain speed it might be worth investigating an optimized implementation of this data structure, such as a singly linked list with a memory pool (SmallVector was tried but slow and SparseSet is not applicable).

Definition at line 190 of file ScheduleDAGInstrs.h.

Constructor & Destructor Documentation

◆ ScheduleDAGInstrs()

ScheduleDAGInstrs::ScheduleDAGInstrs	(	MachineFunction &	mf,
		const MachineLoopInfo *	mli,
		bool	RemoveKillFlags = `false`
	)

explicit

Definition at line 112 of file ScheduleDAGInstrs.cpp.

References DbgValues, llvm::MachineFunction::getSubtarget(), llvm::TargetSchedModel::init(), SchedModel, and llvm::ARM_MB::ST.

◆ ~ScheduleDAGInstrs()

llvm::ScheduleDAGInstrs::~ScheduleDAGInstrs ( )

overridedefault

Member Function Documentation

◆ addBarrierChain()

void ScheduleDAGInstrs::addBarrierChain ( Value2SUsMap & map )

protected

Adds barrier chain edges from all SUs in map, and then clear the map.

This is equivalent to insertBarrierChain(), but optimized for the common case where the new BarrierChain (a global memory object) has a higher NodeNum than all SUs in map. It is assumed BarrierChain has been set before calling this.

Definition at line 660 of file ScheduleDAGInstrs.cpp.

References assert(), BarrierChain, and I.

Referenced by buildSchedGraph().

◆ addChainDependencies() [1/3]

void llvm::ScheduleDAGInstrs::addChainDependencies	(	SUnit *	SU,
		SUList &	SUs,
		unsigned	Latency
	)

inlineprotected

Adds dependencies as needed from all SUs in list to SU.

Definition at line 210 of file ScheduleDAGInstrs.h.

References llvm::VReg2SUnit::SU.

Referenced by addChainDependencies(), and buildSchedGraph().

◆ addChainDependencies() [2/3]

void ScheduleDAGInstrs::addChainDependencies	(	SUnit *	SU,
		Value2SUsMap &	Val2SUsMap
	)

protected

Adds dependencies as needed from all SUs in map, to SU.

Definition at line 644 of file ScheduleDAGInstrs.cpp.

References addChainDependencies(), and I.

◆ addChainDependencies() [3/3]

void ScheduleDAGInstrs::addChainDependencies	(	SUnit *	SU,
		Value2SUsMap &	Val2SUsMap,
		ValueType	V
	)

protected

Adds dependencies as needed to SU, from all SUs mapped to V.

Definition at line 651 of file ScheduleDAGInstrs.cpp.

References addChainDependencies(), llvm::MapVector< KeyT, ValueT, MapType, VectorType >::end(), llvm::MapVector< KeyT, ValueT, MapType, VectorType >::find(), and llvm::ScheduleDAGInstrs::Value2SUsMap::getTrueMemOrderLatency().

◆ addChainDependency()

void ScheduleDAGInstrs::addChainDependency	(	SUnit *	SUa,
		SUnit *	SUb,
		unsigned	Latency = `0`
	)

protected

Adds a chain edge between SUa and SUb, but only if both AliasAnalysis and Target fail to deny the dependency.

Definition at line 522 of file ScheduleDAGInstrs.cpp.

References AAForDep, llvm::SUnit::addPred(), llvm::SUnit::getInstr(), llvm::MachineInstr::mayAlias(), llvm::SDep::MayAliasMem, llvm::SDep::setLatency(), and UseTBAA.

◆ addPhysRegDataDeps()

void ScheduleDAGInstrs::addPhysRegDataDeps	(	SUnit *	SU,
		unsigned	OperIdx
	)

protected

MO is an operand of SU's instruction that defines a physical register.

Adds data dependencies from SU to any uses of the physical register.

Definition at line 230 of file ScheduleDAGInstrs.cpp.

References llvm::SUnit::addPred(), llvm::SDep::Artificial, assert(), llvm::TargetSchedModel::computeOperandLatency(), llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::contains(), llvm::SDep::Data, llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::end(), llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::find(), llvm::MachineInstr::getDesc(), llvm::SUnit::getInstr(), llvm::MCInstrDesc::getNumOperands(), llvm::MachineInstr::getOperand(), llvm::MachineOperand::getReg(), llvm::MachineFunction::getSubtarget(), llvm::MCInstrDesc::hasImplicitDefOfPhysReg(), llvm::MCInstrDesc::hasImplicitUseOfPhysReg(), llvm::SUnit::hasPhysRegDefs, I, llvm::MachineOperand::isDef(), llvm::MCRegAliasIterator::isValid(), llvm::ScheduleDAG::MF, SchedModel, llvm::SDep::setLatency(), llvm::ARM_MB::ST, llvm::ScheduleDAG::TRI, and Uses.

Referenced by addPhysRegDeps().

◆ addPhysRegDeps()

void ScheduleDAGInstrs::addPhysRegDeps	(	SUnit *	SU,
		unsigned	OperIdx
	)

protected

Adds register dependencies (data, anti, and output) from this SUnit to following instructions in the same scheduling region that depend the physical register referenced at OperIdx.

Definition at line 285 of file ScheduleDAGInstrs.cpp.

References addPhysRegDataDeps(), llvm::SUnit::addPred(), llvm::SDep::Anti, B, llvm::TargetSchedModel::computeOutputLatency(), llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::contains(), Defs, llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::end(), llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::equal_range(), llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::erase(), llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::eraseAll(), llvm::ScheduleDAG::ExitSU, llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::find(), llvm::SUnit::getInstr(), llvm::MachineInstr::getOperand(), llvm::MachineOperand::getReg(), llvm::SUnit::hasPhysRegUses, I, llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::insert(), llvm::SUnit::isCall, llvm::MachineRegisterInfo::isConstantPhysReg(), llvm::MachineOperand::isDead(), llvm::MachineOperand::isDef(), llvm::MachineOperand::isUse(), llvm::MCRegAliasIterator::isValid(), Kind, MI, llvm::ScheduleDAG::MRI, llvm::SDep::Output, P, Reg, llvm::MachineInstr::registerDefIsDead(), RemoveKillFlags, SchedModel, llvm::MachineOperand::setIsKill(), llvm::SDep::setLatency(), SubReg, llvm::ScheduleDAG::TRI, and Uses.

Referenced by buildSchedGraph().

◆ addSchedBarrierDeps()

void ScheduleDAGInstrs::addSchedBarrierDeps ( )

Adds dependencies from instructions in the current list of instructions being scheduled to scheduling barrier.

We want to make sure instructions which define registers that are either used by the terminator or are live-out are properly scheduled. This is especially important when the definition latency of the return value(s) are too high to be hidden by the branch or when the liveout registers used by instructions in the fallthrough block.

Definition at line 201 of file ScheduleDAGInstrs.cpp.

References addVRegUseDeps(), BB, llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::contains(), llvm::MachineBasicBlock::end(), llvm::ScheduleDAG::ExitSU, llvm::MachineInstr::getOperandNo(), llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::insert(), llvm::MachineInstr::isBarrier(), llvm::MachineInstr::isCall(), llvm::TargetRegisterInfo::isPhysicalRegister(), llvm::TargetRegisterInfo::isVirtualRegister(), llvm::MachineInstr::operands(), Reg, RegionEnd, llvm::SUnit::setInstr(), llvm::MachineBasicBlock::successors(), and Uses.

Referenced by buildSchedGraph().

◆ addVRegDefDeps()

void ScheduleDAGInstrs::addVRegDefDeps	(	SUnit *	SU,
		unsigned	OperIdx
	)

protected

Adds register output and data dependencies from this SUnit to instructions that occur later in the same scheduling region if they read from or write to the virtual register defined at OperIdx.

TODO: Hoist loop induction variable increments. This has to be reevaluated. Generally, IV scheduling should be done before coalescing.

Definition at line 382 of file ScheduleDAGInstrs.cpp.

References llvm::SUnit::addPred(), llvm::TargetSubtargetInfo::adjustSchedDependency(), llvm::LaneBitmask::any(), assert(), llvm::TargetSchedModel::computeOperandLatency(), llvm::TargetSchedModel::computeOutputLatency(), CurrentVRegDefs, CurrentVRegUses, llvm::SDep::Data, E, llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::end(), llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::erase(), llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::find(), llvm::LaneBitmask::getAll(), llvm::SUnit::getInstr(), getLaneMaskForMO(), llvm::MachineInstr::getOperand(), llvm::MachineOperand::getReg(), llvm::MachineOperand::getSubReg(), llvm::MachineFunction::getSubtarget(), llvm::MachineRegisterInfo::hasOneDef(), I, llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::insert(), llvm::MachineOperand::isDead(), llvm::MachineOperand::isUndef(), llvm::make_range(), llvm::ScheduleDAG::MF, MI, llvm::ScheduleDAG::MRI, llvm::SDep::Output, Reg, SchedModel, llvm::MachineOperand::setIsUndef(), llvm::SDep::setLatency(), llvm::ARM_MB::ST, and TrackLaneMasks.

Referenced by buildSchedGraph().

◆ addVRegUseDeps()

void ScheduleDAGInstrs::addVRegUseDeps	(	SUnit *	SU,
		unsigned	OperIdx
	)

protected

Adds a register data dependency if the instruction that defines the virtual register used at OperIdx is mapped to an SUnit.

Add a register antidependency from this SUnit to instructions that occur later in the same scheduling region if they write the virtual register.

TODO: Handle ExitSU "uses" properly.

Definition at line 491 of file ScheduleDAGInstrs.cpp.

References llvm::SDep::Anti, CurrentVRegDefs, CurrentVRegUses, llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::end(), llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::find(), llvm::LaneBitmask::getAll(), llvm::SUnit::getInstr(), getLaneMaskForMO(), llvm::MachineInstr::getOperand(), llvm::MachineOperand::getReg(), llvm::SparseMultiSet< ValueT, KeyFunctorT, SparseT >::insert(), llvm::make_range(), MI, Reg, and TrackLaneMasks.

Referenced by addSchedBarrierDeps(), and buildSchedGraph().

◆ begin()

MachineBasicBlock::iterator llvm::ScheduleDAGInstrs::begin ( ) const

inline

Returns an iterator to the top of the current scheduling region.

Definition at line 267 of file ScheduleDAGInstrs.h.

Referenced by llvm::createCopyConstrainDAGMutation(), enterRegion(), llvm::GCNScheduleDAGMILive::finalizeSchedule(), llvm::GCNScheduleDAGMILive::schedule(), and llvm::SIScheduleDAGMI::schedule().

◆ buildSchedGraph()

void ScheduleDAGInstrs::buildSchedGraph	(	AliasAnalysis *	AA,
		RegPressureTracker *	RPTracker = `nullptr`,
		PressureDiffs *	PDiffs = `nullptr`,
		LiveIntervals *	LIS = `nullptr`,
		bool	TrackLaneMasks = `false`
	)

Builds SUnits for the current region.

If RPTracker is non-null, compute register pressure as a side effect. The DAG builder is an efficient place to do it because it already visits operands.

Definition at line 704 of file ScheduleDAGInstrs.cpp.

Referenced by llvm::GCNIterativeScheduler::BuildDAG::BuildDAG(), and llvm::DefaultVLIWScheduler::schedule().

◆ doMBBSchedRegionsTopDown()

virtual bool llvm::ScheduleDAGInstrs::doMBBSchedRegionsTopDown ( ) const

inlinevirtual

If this method returns true, handling of the scheduling regions themselves (in case of a scheduling boundary in MBB) will be done beginning with the topmost region of MBB.

Reimplemented in llvm::ScheduleDAGMI.

Definition at line 281 of file ScheduleDAGInstrs.h.

References llvm::sys::path::begin(), and llvm::sys::path::end().

Referenced by getSchedRegions().

◆ dump()

void ScheduleDAGInstrs::dump ( ) const

overridevirtual

Implements llvm::ScheduleDAG.

Definition at line 1120 of file ScheduleDAGInstrs.cpp.

References llvm::ScheduleDAG::dumpNodeAll(), llvm::ScheduleDAG::EntrySU, llvm::ScheduleDAG::ExitSU, llvm::SUnit::getInstr(), and llvm::ScheduleDAG::SUnits.

Referenced by llvm::ScheduleDAGInstrs::Value2SUsMap::getTrueMemOrderLatency(), and llvm::VLIWPacketizerList::PacketizeMIs().

◆ dumpNode()

void ScheduleDAGInstrs::dumpNode ( const SUnit & SU ) const

overridevirtual

Implements llvm::ScheduleDAG.

Definition at line 1112 of file ScheduleDAGInstrs.cpp.

References llvm::dbgs(), llvm::MachineInstr::dump(), llvm::ScheduleDAG::dumpNodeName(), and llvm::SUnit::getInstr().

Referenced by llvm::R600SchedStrategy::pickNode(), llvm::R600SchedStrategy::releaseBottomNode(), llvm::R600SchedStrategy::releaseTopNode(), and llvm::GenericScheduler::reschedulePhysReg().

◆ end()

MachineBasicBlock::iterator llvm::ScheduleDAGInstrs::end ( ) const

inline

Returns an iterator to the bottom of the current scheduling region.

Definition at line 270 of file ScheduleDAGInstrs.h.

References MI.

Referenced by llvm::ScheduleDAGInstrs::Value2SUsMap::clearList(), llvm::createCopyConstrainDAGMutation(), enterRegion(), llvm::GCNScheduleDAGMILive::finalizeSchedule(), and llvm::GCNScheduleDAGMILive::schedule().

◆ enterRegion()

void ScheduleDAGInstrs::enterRegion	(	MachineBasicBlock *	bb,
		MachineBasicBlock::iterator	begin,
		MachineBasicBlock::iterator	end,
		unsigned	regioninstrs
	)

virtual

Initialize the DAG and common scheduler state for a new scheduling region.

This does not actually create the DAG, only clears it. The scheduling driver may call BuildSchedGraph multiple times per scheduling region.

Reimplemented in llvm::ScheduleDAGMILive, llvm::ScheduleDAGMI, and llvm::GCNIterativeScheduler.

Definition at line 187 of file ScheduleDAGInstrs.cpp.

References assert(), BB, begin(), end(), NumRegionInstrs, RegionBegin, and RegionEnd.

Referenced by llvm::ScheduleDAGMI::enterRegion(), getSchedRegions(), INITIALIZE_PASS(), and llvm::VLIWPacketizerList::PacketizeMIs().

◆ exitRegion()

void ScheduleDAGInstrs::exitRegion ( )

virtual

Called when the scheduler has finished scheduling the current region.

Definition at line 197 of file ScheduleDAGInstrs.cpp.

Referenced by llvm::GCNScheduleDAGMILive::finalizeSchedule(), getSchedRegions(), INITIALIZE_PASS(), and llvm::VLIWPacketizerList::PacketizeMIs().

◆ finalizeSchedule()

virtual void llvm::ScheduleDAGInstrs::finalizeSchedule ( )

inlinevirtual

Allow targets to perform final scheduling actions at the level of the whole MachineFunction.

By default does nothing.

Reimplemented in llvm::GCNScheduleDAGMILive, and llvm::GCNIterativeScheduler.

Definition at line 328 of file ScheduleDAGInstrs.h.

References llvm::dump().

Referenced by getSchedRegions().

◆ finishBlock()

void ScheduleDAGInstrs::finishBlock ( )

virtual

Cleans up after scheduling in the given block.

Reimplemented in llvm::ScheduleDAGMI, and llvm::SwingSchedulerDAG.

Definition at line 182 of file ScheduleDAGInstrs.cpp.

References BB.

Referenced by llvm::SwingSchedulerDAG::finishBlock(), llvm::ScheduleDAGMI::finishBlock(), getSchedRegions(), INITIALIZE_PASS(), and llvm::VLIWPacketizerList::PacketizeMIs().

◆ fixupKills()

void ScheduleDAGInstrs::fixupKills ( MachineBasicBlock & MBB )

Fixes register kill flags that scheduling has made invalid.

Definition at line 1060 of file ScheduleDAGInstrs.cpp.

References llvm::LivePhysRegs::addLiveOuts(), llvm::dbgs(), llvm::MachineOperand::getReg(), I, llvm::LivePhysRegs::init(), llvm::MachineOperand::isDef(), llvm::MachineOperand::isReg(), llvm::MachineOperand::isRegMask(), LiveRegs, LLVM_DEBUG, llvm::make_range(), MI, llvm::ScheduleDAG::MRI, llvm::RISCVFenceField::O, llvm::printMBBReference(), llvm::MachineBasicBlock::rbegin(), Reg, llvm::LivePhysRegs::removeReg(), llvm::LivePhysRegs::removeRegsInMask(), llvm::MachineBasicBlock::rend(), toggleKills(), and llvm::ScheduleDAG::TRI.

Referenced by getSchedRegions().

◆ getDAGName()

std::string ScheduleDAGInstrs::getDAGName ( ) const

overridevirtual

Returns a label for the region of code covered by the DAG.

Return the basic block label.

It is not necessarilly unique because a block contains multiple scheduling regions. But it is fine for visualization.

Implements llvm::ScheduleDAG.

Definition at line 1145 of file ScheduleDAGInstrs.cpp.

References BB, and llvm::MachineBasicBlock::getFullName().

◆ getGraphNodeLabel()

std::string ScheduleDAGInstrs::getGraphNodeLabel ( const SUnit * SU ) const

overridevirtual

Returns a label for a DAG node that points to an instruction.

Implements llvm::ScheduleDAG.

Definition at line 1131 of file ScheduleDAGInstrs.cpp.

References llvm::ScheduleDAG::EntrySU, llvm::ScheduleDAG::ExitSU, llvm::SUnit::getInstr(), llvm::MachineInstr::print(), and llvm::raw_string_ostream::str().

◆ getLaneMaskForMO()

LaneBitmask ScheduleDAGInstrs::getLaneMaskForMO ( const MachineOperand & MO ) const

protected

Returns a mask for which lanes get read/written by the given (register) machine operand.

Definition at line 362 of file ScheduleDAGInstrs.cpp.

References llvm::LaneBitmask::getAll(), llvm::TargetRegisterClass::getLaneMask(), llvm::MachineOperand::getReg(), llvm::MachineRegisterInfo::getRegClass(), llvm::MachineOperand::getSubReg(), llvm::TargetRegisterInfo::getSubRegIndexLaneMask(), llvm::TargetRegisterClass::HasDisjunctSubRegs, llvm::ScheduleDAG::MRI, Reg, SubReg, and llvm::ScheduleDAG::TRI.

Referenced by addVRegDefDeps(), and addVRegUseDeps().

◆ getSchedClass()

const MCSchedClassDesc* llvm::ScheduleDAGInstrs::getSchedClass ( SUnit * SU ) const

inline

Resolves and cache a resolved scheduling class for an SUnit.

Definition at line 260 of file ScheduleDAGInstrs.h.

References llvm::SUnit::getInstr(), llvm::TargetSchedModel::hasInstrSchedModel(), llvm::TargetSchedModel::resolveSchedClass(), and llvm::SUnit::SchedClass.

Referenced by llvm::SchedBoundary::bumpNode(), llvm::SchedBoundary::checkHazard(), llvm::SchedRemainder::init(), llvm::GenericSchedulerBase::SchedCandidate::initResourceDelta(), and initSUnits().

◆ getSchedModel()

const TargetSchedModel* llvm::ScheduleDAGInstrs::getSchedModel ( ) const

inline

Gets the machine model for instruction scheduling.

Definition at line 257 of file ScheduleDAGInstrs.h.

Referenced by llvm::GenericScheduler::initialize(), and llvm::PostGenericScheduler::initialize().

◆ getSUnit()

SUnit * llvm::ScheduleDAGInstrs::getSUnit ( MachineInstr * MI ) const

inline

Returns an existing SUnit for this MI, or nullptr.

Definition at line 376 of file ScheduleDAGInstrs.h.

References I.

Referenced by llvm::createCopyConstrainDAGMutation(), llvm::SMSchedule::finalizeSchedule(), and llvm::SMSchedule::isLoopCarried().

◆ initSUnits()

void ScheduleDAGInstrs::initSUnits ( )

protected

Creates an SUnit for each real instruction, numbered in top-down topological order.

The instruction order A < B, implies that no edge exists from B to A.

Map each real instruction to its SUnit.

After initSUnits, the SUnits vector cannot be resized and the scheduler may hang onto SUnit pointers. We may relax this in the future by using SUnit IDs instead of pointers.

MachineScheduler relies on initSUnits numbering the nodes by their order in the original instruction list.

Definition at line 543 of file ScheduleDAGInstrs.cpp.

References llvm::MCProcResourceDesc::BufferSize, llvm::SUnit::getInstr(), llvm::TargetSchedModel::getProcResource(), getSchedClass(), llvm::TargetSchedModel::getWriteProcResBegin(), llvm::TargetSchedModel::getWriteProcResEnd(), llvm::TargetSchedModel::hasInstrSchedModel(), llvm::SUnit::hasReservedResource, llvm::SUnit::isCall, llvm::SUnit::isCommutable, llvm::SUnit::isUnbuffered, llvm::SUnit::Latency, llvm::make_range(), MI, MISUnitMap, newSUnit(), NumRegionInstrs, RegionBegin, RegionEnd, llvm::PPCISD::SC, SchedModel, and llvm::ScheduleDAG::SUnits.

Referenced by buildSchedGraph().

◆ insertBarrierChain()

void ScheduleDAGInstrs::insertBarrierChain ( Value2SUsMap & map )

protected

Inserts a barrier chain in a huge region, far below current SU.

Adds barrier chain edges from all SUs in map with higher NodeNums than this new BarrierChain, and remove them from map. It is assumed BarrierChain has been set before calling this.

Definition at line 671 of file ScheduleDAGInstrs.cpp.

References assert(), BarrierChain, llvm::MapVector< KeyT, ValueT, MapType, VectorType >::begin(), llvm::MapVector< KeyT, ValueT, MapType, VectorType >::end(), I, llvm::SUnit::NodeNum, llvm::ScheduleDAGInstrs::Value2SUsMap::reComputeSize(), and llvm::MapVector< KeyT, ValueT, MapType, VectorType >::remove_if().

Referenced by reduceHugeMemNodeMaps().

◆ newSUnit()

SUnit * llvm::ScheduleDAGInstrs::newSUnit ( MachineInstr * MI )

inline

Creates a new SUnit and return a ptr to it.

Definition at line 365 of file ScheduleDAGInstrs.h.

References assert().

Referenced by initSUnits().

◆ reduceHugeMemNodeMaps()

void ScheduleDAGInstrs::reduceHugeMemNodeMaps	(	Value2SUsMap &	stores,
		Value2SUsMap &	loads,
		unsigned	N
	)

protected

Reduces maps in FIFO order, by N SUs.

This is better than turning every Nth memory SU into BarrierChain in buildSchedGraph(), since it avoids unnecessary edges between seen SUs above the new BarrierChain, and those below it.

Definition at line 997 of file ScheduleDAGInstrs.cpp.

References llvm::SUnit::addPredBarrier(), assert(), BarrierChain, llvm::dbgs(), llvm::ScheduleDAGInstrs::Value2SUsMap::dump(), I, insertBarrierChain(), LLVM_DEBUG, N, llvm::SUnit::NodeNum, llvm::ScheduleDAGInstrs::Value2SUsMap::size(), llvm::sort(), and llvm::ScheduleDAG::SUnits.

Referenced by buildSchedGraph().

◆ schedule()

virtual void llvm::ScheduleDAGInstrs::schedule ( )

pure virtual

Orders nodes according to selected style.

Typically, a scheduling algorithm will implement schedule() without overriding enterRegion() or exitRegion().

Implemented in llvm::ScheduleDAGMILive, llvm::SIScheduleDAGMI, llvm::ScheduleDAGMI, llvm::SwingSchedulerDAG, llvm::DefaultVLIWScheduler, llvm::GCNScheduleDAGMILive, llvm::VLIWMachineScheduler, and llvm::GCNIterativeScheduler.

Referenced by getSchedRegions().

◆ startBlock()

void ScheduleDAGInstrs::startBlock ( MachineBasicBlock * BB )

virtual

Prepares to perform scheduling in the given block.

Reimplemented in llvm::ScheduleDAGMI.

Definition at line 178 of file ScheduleDAGInstrs.cpp.

References BB.

Referenced by getSchedRegions(), INITIALIZE_PASS(), llvm::VLIWPacketizerList::PacketizeMIs(), and llvm::ScheduleDAGMI::startBlock().

◆ startBlockForKills()

void llvm::ScheduleDAGInstrs::startBlockForKills ( MachineBasicBlock * BB )

protected

Initializes register live-range state for updating kills.

PostRA helper for rewriting kill flags.

◆ toggleKillFlag()

void llvm::ScheduleDAGInstrs::toggleKillFlag	(	MachineInstr &	MI,
		MachineOperand &	MO
	)

protected

Toggles a register operand kill flag.

Other adjustments may be made to the instruction if necessary. Return true if the operand has been deleted, false if not.

Member Data Documentation

◆ AAForDep

AliasAnalysis* llvm::ScheduleDAGInstrs::AAForDep = nullptr

protected

Definition at line 177 of file ScheduleDAGInstrs.h.

Referenced by addChainDependency(), and buildSchedGraph().

◆ BarrierChain

SUnit* llvm::ScheduleDAGInstrs::BarrierChain = nullptr

protected

Remember a generic side-effecting instruction as we proceed.

No other SU ever gets scheduled around it (except in the special case of a huge region that gets reduced).

Definition at line 182 of file ScheduleDAGInstrs.h.

Referenced by addBarrierChain(), buildSchedGraph(), insertBarrierChain(), and reduceHugeMemNodeMaps().

◆ BB

MachineBasicBlock* llvm::ScheduleDAGInstrs::BB

protected

The block in which to insert instructions.

Definition at line 145 of file ScheduleDAGInstrs.h.

Referenced by addSchedBarrierDeps(), llvm::GCNIterativeScheduler::BuildDAG::BuildDAG(), enterRegion(), finishBlock(), getDAGName(), llvm::GCNIterativeScheduler::OverrideLegacyStrategy::OverrideLegacyStrategy(), llvm::GCNScheduleDAGMILive::schedule(), llvm::GCNIterativeScheduler::scheduleRegion(), and startBlock().

◆ CanHandleTerminators

bool llvm::ScheduleDAGInstrs::CanHandleTerminators = false

protected

The standard DAG builder does not normally include terminators as DAG nodes because it does not create the necessary dependencies to prevent reordering.

A specialized scheduler can override TargetInstrInfo::isSchedulingBoundary then enable this flag to indicate it has taken responsibility for scheduling the terminator correctly.

Definition at line 136 of file ScheduleDAGInstrs.h.

Referenced by buildSchedGraph(), and llvm::DefaultVLIWScheduler::DefaultVLIWScheduler().

◆ CurrentVRegDefs

VReg2SUnitMultiMap llvm::ScheduleDAGInstrs::CurrentVRegDefs

protected

Tracks the last instruction(s) in this region defining each virtual register.

There may be multiple current definitions for a register with disjunct lanemasks.

Definition at line 173 of file ScheduleDAGInstrs.h.

Referenced by addVRegDefDeps(), addVRegUseDeps(), and buildSchedGraph().

◆ CurrentVRegUses

VReg2SUnitOperIdxMultiMap llvm::ScheduleDAGInstrs::CurrentVRegUses

protected

Tracks the last instructions in this region using each virtual register.

Definition at line 175 of file ScheduleDAGInstrs.h.

Referenced by addVRegDefDeps(), addVRegUseDeps(), and buildSchedGraph().

◆ DbgValues

DbgValueVector llvm::ScheduleDAGInstrs::DbgValues

protected

Remember instruction that precedes DBG_VALUE.

These are generated by buildSchedGraph but persist so they can be referenced when emitting the final schedule.

Definition at line 243 of file ScheduleDAGInstrs.h.

Referenced by buildSchedGraph(), llvm::GCNIterativeScheduler::detachSchedule(), and ScheduleDAGInstrs().

◆ Defs

Reg2SUnitsMap llvm::ScheduleDAGInstrs::Defs

protected

Defs, Uses - Remember where defs and uses of each register are as we iterate upward through the instructions.

This is allocated here instead of inside BuildSchedGraph to avoid the need for it to be initialized and destructed for each block.

Definition at line 167 of file ScheduleDAGInstrs.h.

Referenced by addPhysRegDeps(), and buildSchedGraph().

◆ FirstDbgValue

MachineInstr* llvm::ScheduleDAGInstrs::FirstDbgValue = nullptr

protected

Definition at line 244 of file ScheduleDAGInstrs.h.

Referenced by buildSchedGraph(), and llvm::GCNIterativeScheduler::detachSchedule().

◆ LiveRegs

LivePhysRegs llvm::ScheduleDAGInstrs::LiveRegs

protected

Set of live physical registers for updating kill flags.

Definition at line 247 of file ScheduleDAGInstrs.h.

Referenced by fixupKills().

◆ MFI

const MachineFrameInfo& llvm::ScheduleDAGInstrs::MFI

protected

Definition at line 122 of file ScheduleDAGInstrs.h.

Referenced by buildSchedGraph(), llvm::GCNIterativeScheduler::scheduleILP(), llvm::GCNIterativeScheduler::scheduleLegacyMaxOccupancy(), llvm::GCNIterativeScheduler::scheduleMinReg(), and llvm::GCNIterativeScheduler::tryMaximizeOccupancy().

◆ MISUnitMap

DenseMap<MachineInstr*, SUnit*> llvm::ScheduleDAGInstrs::MISUnitMap

protected

After calling BuildSchedGraph, each machine instruction in the current scheduling region is mapped to an SUnit.

Definition at line 158 of file ScheduleDAGInstrs.h.

Referenced by buildSchedGraph(), and initSUnits().

◆ MLI

const MachineLoopInfo* llvm::ScheduleDAGInstrs::MLI

protected

Definition at line 121 of file ScheduleDAGInstrs.h.

◆ NumRegionInstrs

unsigned llvm::ScheduleDAGInstrs::NumRegionInstrs

protected

Instructions in this region (distance(RegionBegin, RegionEnd)).

Definition at line 154 of file ScheduleDAGInstrs.h.

Referenced by enterRegion(), llvm::GCNScheduleDAGMILive::finalizeSchedule(), initSUnits(), and llvm::GCNScheduleDAGMILive::schedule().

◆ RegionBegin

MachineBasicBlock::iterator llvm::ScheduleDAGInstrs::RegionBegin

protected

The beginning of the range to be scheduled.

Definition at line 148 of file ScheduleDAGInstrs.h.

Referenced by buildSchedGraph(), enterRegion(), llvm::GCNScheduleDAGMILive::finalizeSchedule(), initSUnits(), llvm::GCNIterativeScheduler::OverrideLegacyStrategy::restoreOrder(), llvm::GCNIterativeScheduler::schedule(), llvm::GCNScheduleDAGMILive::schedule(), llvm::GCNIterativeScheduler::OverrideLegacyStrategy::schedule(), llvm::SIScheduleDAGMI::schedule(), and llvm::GCNIterativeScheduler::scheduleRegion().

◆ RegionEnd

MachineBasicBlock::iterator llvm::ScheduleDAGInstrs::RegionEnd

protected

The end of the range to be scheduled.

Definition at line 151 of file ScheduleDAGInstrs.h.

Referenced by addSchedBarrierDeps(), buildSchedGraph(), enterRegion(), llvm::GCNScheduleDAGMILive::finalizeSchedule(), initSUnits(), llvm::GCNIterativeScheduler::OverrideLegacyStrategy::restoreOrder(), llvm::GCNIterativeScheduler::schedule(), llvm::GCNScheduleDAGMILive::schedule(), llvm::GCNIterativeScheduler::OverrideLegacyStrategy::schedule(), and llvm::GCNIterativeScheduler::scheduleRegion().

◆ RemoveKillFlags

bool llvm::ScheduleDAGInstrs::RemoveKillFlags

protected

True if the DAG builder should remove kill flags (in preparation for rescheduling).

Definition at line 129 of file ScheduleDAGInstrs.h.

Referenced by addPhysRegDeps().

◆ SchedModel

TargetSchedModel llvm::ScheduleDAGInstrs::SchedModel

protected

TargetSchedModel provides an interface to the machine model.

Definition at line 125 of file ScheduleDAGInstrs.h.

Referenced by addPhysRegDataDeps(), addPhysRegDeps(), addVRegDefDeps(), initSUnits(), and ScheduleDAGInstrs().

◆ TrackLaneMasks

bool llvm::ScheduleDAGInstrs::TrackLaneMasks = false

protected

Whether lane masks should get tracked.

Definition at line 139 of file ScheduleDAGInstrs.h.

Referenced by addVRegDefDeps(), addVRegUseDeps(), and buildSchedGraph().

◆ UnknownValue

UndefValue* llvm::ScheduleDAGInstrs::UnknownValue

protected

For an unanalyzable memory access, this Value is used in maps.

Definition at line 236 of file ScheduleDAGInstrs.h.

Referenced by buildSchedGraph().

◆ Uses

Reg2SUnitsMap llvm::ScheduleDAGInstrs::Uses

protected

Definition at line 168 of file ScheduleDAGInstrs.h.

Referenced by addPhysRegDataDeps(), addPhysRegDeps(), addSchedBarrierDeps(), and buildSchedGraph().

The documentation for this class was generated from the following files:

include/llvm/CodeGen/ScheduleDAGInstrs.h
lib/CodeGen/ScheduleDAGInstrs.cpp

Classes

Public Types

Public Member Functions

Protected Types

Protected Member Functions

Protected Attributes

Additional Inherited Members

Detailed Description

Member Typedef Documentation

◆ DbgValueVector

◆ SUList

Constructor & Destructor Documentation

◆ ScheduleDAGInstrs()

◆ ~ScheduleDAGInstrs()

Member Function Documentation

◆ addBarrierChain()

◆ addChainDependencies() [1/3]

◆ addChainDependencies() [2/3]

◆ addChainDependencies() [3/3]

◆ addChainDependency()

◆ addPhysRegDataDeps()

◆ addPhysRegDeps()

◆ addSchedBarrierDeps()

◆ addVRegDefDeps()

◆ addVRegUseDeps()

◆ begin()

◆ buildSchedGraph()

◆ doMBBSchedRegionsTopDown()

◆ dump()

◆ dumpNode()

◆ end()

◆ enterRegion()

◆ exitRegion()

◆ finalizeSchedule()

◆ finishBlock()

◆ fixupKills()

◆ getDAGName()

◆ getGraphNodeLabel()

◆ getLaneMaskForMO()

◆ getSchedClass()

◆ getSchedModel()

◆ getSUnit()

◆ initSUnits()

◆ insertBarrierChain()

◆ newSUnit()

◆ reduceHugeMemNodeMaps()

◆ schedule()

◆ startBlock()

◆ startBlockForKills()

◆ toggleKillFlag()

Member Data Documentation

◆ AAForDep

◆ BarrierChain

◆ BB

◆ CanHandleTerminators

◆ CurrentVRegDefs

◆ CurrentVRegUses

◆ DbgValues

◆ Defs

◆ FirstDbgValue

◆ LiveRegs

◆ MFI

◆ MISUnitMap

◆ MLI

◆ NumRegionInstrs

◆ RegionBegin

◆ RegionEnd

◆ RemoveKillFlags

◆ SchedModel

◆ TrackLaneMasks

◆ UnknownValue

◆ Uses