This class builds the dependence graph for the instructions in a loop, and attempts to schedule the instructions using the SMS algorithm. More...

#include "llvm/CodeGen/MachinePipeliner.h"

Inheritance diagram for llvm::SwingSchedulerDAG:

[legend]

Collaboration diagram for llvm::SwingSchedulerDAG:

[legend]

Public Member Functions
	SwingSchedulerDAG (MachinePipeliner &P, MachineLoop &L, LiveIntervals &lis, const RegisterClassInfo &rci)

void	schedule () override
	We override the schedule function in ScheduleDAGInstrs to implement the scheduling part of the Swing Modulo Scheduling algorithm. More...

void	finishBlock () override
	Clean up after the software pipeliner runs. More...

bool	hasNewSchedule ()
	Return true if the loop kernel has been scheduled. More...

int	getASAP (SUnit *Node)
	Return the earliest time an instruction may be scheduled. More...

int	getALAP (SUnit *Node)
	Return the latest time an instruction my be scheduled. More...

int	getMOV (SUnit *Node)
	The mobility function, which the number of slots in which an instruction may be scheduled. More...

unsigned	getDepth (SUnit *Node)
	The depth, in the dependence graph, for a node. More...

int	getZeroLatencyDepth (SUnit *Node)
	The maximum unweighted length of a path from an arbitrary node to the given node in which each edge has latency 0. More...

unsigned	getHeight (SUnit *Node)
	The height, in the dependence graph, for a node. More...

int	getZeroLatencyHeight (SUnit *Node)
	The maximum unweighted length of a path from the given node to an arbitrary node in which each edge has latency 0. More...

bool	isBackedge (SUnit *Source, const SDep &Dep)
	Return true if the dependence is a back-edge in the data dependence graph. More...

bool	isLoopCarriedDep (SUnit *Source, const SDep &Dep, bool isSucc=true)
	Return true for an order or output dependence that is loop carried potentially. More...

unsigned	getDistance (SUnit U, SUnit V, const SDep &Dep)
	The distance function, which indicates that operation V of iteration I depends on operations U of iteration I-distance. More...

void	setMII (unsigned mii)
	Set the Minimum Initiation Interval for this schedule attempt. More...

void	applyInstrChange (MachineInstr *MI, SMSchedule &Schedule)
	Apply changes to the instruction if needed. More...

void	fixupRegisterOverlaps (std::deque< SUnit *> &Instrs)
	Attempt to fix the degenerate cases when the instruction serialization causes the register lifetimes to overlap. More...

unsigned	getInstrBaseReg (SUnit *SU)
	Return the new base register that was stored away for the changed instruction. More...

void	addMutation (std::unique_ptr< ScheduleDAGMutation > Mutation)

Public Member Functions inherited from llvm::ScheduleDAGInstrs
	ScheduleDAGInstrs (MachineFunction &mf, const MachineLoopInfo *mli, bool RemoveKillFlags=false)

	~ScheduleDAGInstrs () override=default

const TargetSchedModel *	getSchedModel () const
	Gets the machine model for instruction scheduling. More...

const MCSchedClassDesc *	getSchedClass (SUnit *SU) const
	Resolves and cache a resolved scheduling class for an SUnit. More...

MachineBasicBlock::iterator	begin () const
	Returns an iterator to the top of the current scheduling region. More...

MachineBasicBlock::iterator	end () const
	Returns an iterator to the bottom of the current scheduling region. More...

SUnit *	newSUnit (MachineInstr *MI)
	Creates a new SUnit and return a ptr to it. More...

SUnit *	getSUnit (MachineInstr *MI) const
	Returns an existing SUnit for this MI, or nullptr. More...

virtual bool	doMBBSchedRegionsTopDown () const
	If this method returns true, handling of the scheduling regions themselves (in case of a scheduling boundary in MBB) will be done beginning with the topmost region of MBB. More...

virtual void	startBlock (MachineBasicBlock *BB)
	Prepares to perform scheduling in the given block. More...

virtual void	enterRegion (MachineBasicBlock *bb, MachineBasicBlock::iterator begin, MachineBasicBlock::iterator end, unsigned regioninstrs)
	Initialize the DAG and common scheduler state for a new scheduling region. More...

virtual void	exitRegion ()
	Called when the scheduler has finished scheduling the current region. More...

void	buildSchedGraph (AliasAnalysis AA, RegPressureTracker RPTracker=nullptr, PressureDiffs PDiffs=nullptr, LiveIntervals LIS=nullptr, bool TrackLaneMasks=false)
	Builds SUnits for the current region. More...

void	addSchedBarrierDeps ()
	Adds dependencies from instructions in the current list of instructions being scheduled to scheduling barrier. More...

virtual void	finalizeSchedule ()
	Allow targets to perform final scheduling actions at the level of the whole MachineFunction. More...

void	dumpNode (const SUnit &SU) const override

void	dump () const override

std::string	getGraphNodeLabel (const SUnit *SU) const override
	Returns a label for a DAG node that points to an instruction. More...

std::string	getDAGName () const override
	Returns a label for the region of code covered by the DAG. More...

void	fixupKills (MachineBasicBlock &MBB)
	Fixes register kill flags that scheduling has made invalid. More...

Public Member Functions inherited from llvm::ScheduleDAG
	ScheduleDAG (MachineFunction &mf)

virtual	~ScheduleDAG ()

void	clearDAG ()
	Clears the DAG state (between regions). More...

const MCInstrDesc *	getInstrDesc (const SUnit *SU) const
	Returns the MCInstrDesc of this SUnit. More...

virtual void	viewGraph (const Twine &Name, const Twine &Title)
	Pops up a GraphViz/gv window with the ScheduleDAG rendered using 'dot'. More...

virtual void	viewGraph ()
	Out-of-line implementation with no arguments is handy for gdb. More...

void	dumpNodeName (const SUnit &SU) const

virtual void	addCustomGraphFeatures (GraphWriter< ScheduleDAG *> &) const
	Adds custom features for a visualization of the ScheduleDAG. More...

unsigned	VerifyScheduledDAG (bool isBottomUp)
	Verifies that all SUnits were scheduled and that their state is consistent. More...

Static Public Member Functions
static bool	classof (const ScheduleDAGInstrs *DAG)

Additional Inherited Members
Public Types inherited from llvm::ScheduleDAGInstrs
using	SUList = std::list< SUnit * >
	A list of SUnits, used in Value2SUsMap, during DAG construction. More...

Public Attributes inherited from llvm::ScheduleDAG
const LLVMTargetMachine &	TM
	Target processor. More...

const TargetInstrInfo *	TII
	Target instruction information. More...

const TargetRegisterInfo *	TRI
	Target processor register info. More...

MachineFunction &	MF
	Machine function. More...

MachineRegisterInfo &	MRI
	Virtual/real register map. More...

std::vector< SUnit >	SUnits
	The scheduling units. More...

SUnit	EntrySU
	Special node for the region entry. More...

SUnit	ExitSU
	Special node for the region exit. More...

bool	StressSched

Protected Types inherited from llvm::ScheduleDAGInstrs
using	DbgValueVector = std::vector< std::pair< MachineInstr , MachineInstr > >

Protected Member Functions inherited from llvm::ScheduleDAGInstrs
void	reduceHugeMemNodeMaps (Value2SUsMap &stores, Value2SUsMap &loads, unsigned N)
	Reduces maps in FIFO order, by N SUs. More...

void	addChainDependency (SUnit SUa, SUnit SUb, unsigned Latency=0)
	Adds a chain edge between SUa and SUb, but only if both AliasAnalysis and Target fail to deny the dependency. More...

void	addChainDependencies (SUnit *SU, SUList &SUs, unsigned Latency)
	Adds dependencies as needed from all SUs in list to SU. More...

void	addChainDependencies (SUnit *SU, Value2SUsMap &Val2SUsMap)
	Adds dependencies as needed from all SUs in map, to SU. More...

void	addChainDependencies (SUnit *SU, Value2SUsMap &Val2SUsMap, ValueType V)
	Adds dependencies as needed to SU, from all SUs mapped to V. More...

void	addBarrierChain (Value2SUsMap &map)
	Adds barrier chain edges from all SUs in map, and then clear the map. More...

void	insertBarrierChain (Value2SUsMap &map)
	Inserts a barrier chain in a huge region, far below current SU. More...

void	initSUnits ()
	Creates an SUnit for each real instruction, numbered in top-down topological order. More...

void	addPhysRegDataDeps (SUnit *SU, unsigned OperIdx)
	MO is an operand of SU's instruction that defines a physical register. More...

void	addPhysRegDeps (SUnit *SU, unsigned OperIdx)
	Adds register dependencies (data, anti, and output) from this SUnit to following instructions in the same scheduling region that depend the physical register referenced at OperIdx. More...

void	addVRegDefDeps (SUnit *SU, unsigned OperIdx)
	Adds register output and data dependencies from this SUnit to instructions that occur later in the same scheduling region if they read from or write to the virtual register defined at OperIdx. More...

void	addVRegUseDeps (SUnit *SU, unsigned OperIdx)
	Adds a register data dependency if the instruction that defines the virtual register used at OperIdx is mapped to an SUnit. More...

void	startBlockForKills (MachineBasicBlock *BB)
	Initializes register live-range state for updating kills. More...

void	toggleKillFlag (MachineInstr &MI, MachineOperand &MO)
	Toggles a register operand kill flag. More...

LaneBitmask	getLaneMaskForMO (const MachineOperand &MO) const
	Returns a mask for which lanes get read/written by the given (register) machine operand. More...

Protected Member Functions inherited from llvm::ScheduleDAG
void	dumpNodeAll (const SUnit &SU) const

Protected Attributes inherited from llvm::ScheduleDAGInstrs
const MachineLoopInfo *	MLI

const MachineFrameInfo &	MFI

TargetSchedModel	SchedModel
	TargetSchedModel provides an interface to the machine model. More...

bool	RemoveKillFlags
	True if the DAG builder should remove kill flags (in preparation for rescheduling). More...

bool	CanHandleTerminators = false
	The standard DAG builder does not normally include terminators as DAG nodes because it does not create the necessary dependencies to prevent reordering. More...

bool	TrackLaneMasks = false
	Whether lane masks should get tracked. More...

MachineBasicBlock *	BB
	The block in which to insert instructions. More...

MachineBasicBlock::iterator	RegionBegin
	The beginning of the range to be scheduled. More...

MachineBasicBlock::iterator	RegionEnd
	The end of the range to be scheduled. More...

unsigned	NumRegionInstrs
	Instructions in this region (distance(RegionBegin, RegionEnd)). More...

DenseMap< MachineInstr , SUnit >	MISUnitMap
	After calling BuildSchedGraph, each machine instruction in the current scheduling region is mapped to an SUnit. More...

Reg2SUnitsMap	Defs
	Defs, Uses - Remember where defs and uses of each register are as we iterate upward through the instructions. More...

Reg2SUnitsMap	Uses

VReg2SUnitMultiMap	CurrentVRegDefs
	Tracks the last instruction(s) in this region defining each virtual register. More...

VReg2SUnitOperIdxMultiMap	CurrentVRegUses
	Tracks the last instructions in this region using each virtual register. More...

AliasAnalysis *	AAForDep = nullptr

SUnit *	BarrierChain = nullptr
	Remember a generic side-effecting instruction as we proceed. More...

UndefValue *	UnknownValue
	For an unanalyzable memory access, this Value is used in maps. More...

DbgValueVector	DbgValues
	Remember instruction that precedes DBG_VALUE. More...

MachineInstr *	FirstDbgValue = nullptr

LivePhysRegs	LiveRegs
	Set of live physical registers for updating kill flags. More...

Detailed Description

This class builds the dependence graph for the instructions in a loop, and attempts to schedule the instructions using the SMS algorithm.

Definition at line 107 of file MachinePipeliner.h.

Constructor & Destructor Documentation

◆ SwingSchedulerDAG()

llvm::SwingSchedulerDAG::SwingSchedulerDAG	(	MachinePipeliner &	P,
		MachineLoop &	L,
		LiveIntervals &	lis,
		const RegisterClassInfo &	rci
	)

inline

Definition at line 192 of file MachinePipeliner.h.

References llvm::TargetSubtargetInfo::getSMSMutations(), llvm::MachineFunction::getSubtarget(), and llvm::MachinePipeliner::MF.

Member Function Documentation

◆ addMutation()

void llvm::SwingSchedulerDAG::addMutation ( std::unique_ptr< ScheduleDAGMutation > Mutation )

inline

Definition at line 273 of file MachinePipeliner.h.

◆ applyInstrChange()

void SwingSchedulerDAG::applyInstrChange	(	MachineInstr *	MI,
		SMSchedule &	Schedule
	)

Apply changes to the instruction if needed.

The changes are need to improve the scheduling and depend up on the final schedule.

Definition at line 3004 of file MachinePipeliner.cpp.

References llvm::MachineFunction::CloneMachineInstr(), llvm::SMSchedule::cycleScheduled(), llvm::DenseMapBase< DenseMap< KeyT, ValueT, KeyInfoT, BucketT >, KeyT, ValueT, KeyInfoT, BucketT >::end(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::find(), llvm::TargetInstrInfo::getBaseAndOffsetPosition(), llvm::MachineOperand::getImm(), llvm::MachineInstr::getOperand(), llvm::MachineOperand::getReg(), llvm::MachinePipeliner::MF, llvm::MachineOperand::setImm(), llvm::SUnit::setInstr(), llvm::MachineOperand::setReg(), llvm::SMSchedule::stageScheduled(), and llvm::MachinePipeliner::TII.

Referenced by llvm::SMSchedule::finalizeSchedule().

◆ classof()

static bool llvm::SwingSchedulerDAG::classof ( const ScheduleDAGInstrs * DAG )

inlinestatic

Definition at line 277 of file MachinePipeliner.h.

References llvm::computeDelta(), and Reg.

◆ finishBlock()

void SwingSchedulerDAG::finishBlock ( )

overridevirtual

Clean up after the software pipeliner runs.

Reimplemented from llvm::ScheduleDAGInstrs.

Definition at line 403 of file MachinePipeliner.cpp.

References llvm::MachineFunction::DeleteMachineInstr(), llvm::ScheduleDAGInstrs::finishBlock(), I, and llvm::MachinePipeliner::MF.

◆ fixupRegisterOverlaps()

void SwingSchedulerDAG::fixupRegisterOverlaps ( std::deque< SUnit *> & Instrs )

Attempt to fix the degenerate cases when the instruction serialization causes the register lifetimes to overlap.

For example, p' = store_pi(p, b) = load p, offset In this case p and p' overlap, which means that two registers are needed. Instead, this function changes the load to use p' and updates the offset.

Definition at line 3579 of file MachinePipeliner.cpp.

References llvm::MachineFunction::CloneMachineInstr(), llvm::DenseMapBase< DenseMap< KeyT, ValueT, KeyInfoT, BucketT >, KeyT, ValueT, KeyInfoT, BucketT >::end(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::find(), llvm::TargetInstrInfo::getBaseAndOffsetPosition(), llvm::MachineOperand::getImm(), llvm::SUnit::getInstr(), llvm::MachineInstr::getNumOperands(), llvm::MachineInstr::getOperand(), llvm::MachineOperand::getReg(), llvm::MachineOperand::isReg(), llvm::MachineInstr::isRegTiedToUseOperand(), llvm::MachineOperand::isUse(), llvm::MachinePipeliner::MF, llvm::MachineOperand::setImm(), llvm::SUnit::setInstr(), llvm::MachineOperand::setReg(), and llvm::MachinePipeliner::TII.

Referenced by llvm::SMSchedule::finalizeSchedule().

◆ getALAP()

int llvm::SwingSchedulerDAG::getALAP ( SUnit * Node )

inline

Return the latest time an instruction my be scheduled.

Definition at line 211 of file MachinePipeliner.h.

◆ getASAP()

int llvm::SwingSchedulerDAG::getASAP ( SUnit * Node )

inline

Return the earliest time an instruction may be scheduled.

Definition at line 208 of file MachinePipeliner.h.

◆ getDepth()

unsigned llvm::SwingSchedulerDAG::getDepth ( SUnit * Node )

inline

The depth, in the dependence graph, for a node.

Definition at line 218 of file MachinePipeliner.h.

Referenced by llvm::NodeSet::computeNodeSetInfo().

◆ getDistance()

unsigned llvm::SwingSchedulerDAG::getDistance	(	SUnit *	U,
		SUnit *	V,
		const SDep &	Dep
	)

inline

The distance function, which indicates that operation V of iteration I depends on operations U of iteration I-distance.

Definition at line 248 of file MachinePipeliner.h.

References llvm::SDep::Anti, llvm::SUnit::getInstr(), llvm::SDep::getKind(), and llvm::MachineInstr::isPHI().

Referenced by llvm::SMSchedule::computeStart().

◆ getHeight()

unsigned llvm::SwingSchedulerDAG::getHeight ( SUnit * Node )

inline

The height, in the dependence graph, for a node.

Definition at line 227 of file MachinePipeliner.h.

◆ getInstrBaseReg()

unsigned llvm::SwingSchedulerDAG::getInstrBaseReg ( SUnit * SU )

inline

Return the new base register that was stored away for the changed instruction.

Definition at line 265 of file MachinePipeliner.h.

References llvm::DenseMapBase< DenseMap< KeyT, ValueT, KeyInfoT, BucketT >, KeyT, ValueT, KeyInfoT, BucketT >::end(), and llvm::DenseMapBase< DenseMap< KeyT, ValueT, KeyInfoT, BucketT >, KeyT, ValueT, KeyInfoT, BucketT >::find().

Referenced by llvm::SMSchedule::orderDependence().

◆ getMOV()

int llvm::SwingSchedulerDAG::getMOV ( SUnit * Node )

inline

The mobility function, which the number of slots in which an instruction may be scheduled.

Definition at line 215 of file MachinePipeliner.h.

Referenced by llvm::NodeSet::computeNodeSetInfo().

◆ getZeroLatencyDepth()

int llvm::SwingSchedulerDAG::getZeroLatencyDepth ( SUnit * Node )

inline

The maximum unweighted length of a path from an arbitrary node to the given node in which each edge has latency 0.

Definition at line 222 of file MachinePipeliner.h.

◆ getZeroLatencyHeight()

int llvm::SwingSchedulerDAG::getZeroLatencyHeight ( SUnit * Node )

inline

The maximum unweighted length of a path from the given node to an arbitrary node in which each edge has latency 0.

Definition at line 231 of file MachinePipeliner.h.

◆ hasNewSchedule()

bool llvm::SwingSchedulerDAG::hasNewSchedule ( )

inline

Return true if the loop kernel has been scheduled.

Definition at line 205 of file MachinePipeliner.h.

◆ isBackedge()

bool llvm::SwingSchedulerDAG::isBackedge	(	SUnit *	Source,
		const SDep &	Dep
	)

inline

Return true if the dependence is a back-edge in the data dependence graph.

Since the DAG doesn't contain cycles, we represent a cycle in the graph using an anti dependence from a Phi to an instruction.

Definition at line 238 of file MachinePipeliner.h.

References llvm::SDep::Anti, llvm::SUnit::getInstr(), llvm::SDep::getKind(), llvm::SDep::getSUnit(), llvm::MachineInstr::isPHI(), and llvm::Sched::Source.

Referenced by llvm::SMSchedule::computeStart(), and multipleIterations().

◆ isLoopCarriedDep()

bool SwingSchedulerDAG::isLoopCarriedDep	(	SUnit *	Source,
		const SDep &	Dep,
		bool	isSucc = `true`
	)

Return true for an order or output dependence that is loop carried potentially.

A dependence is loop carried if the destination defines a valu that may be used or defined by the source in a subsequent iteration.

Definition at line 3041 of file MachinePipeliner.cpp.

References assert(), llvm::computeDelta(), D, llvm::tgtok::Def, llvm::TargetInstrInfo::getIncrementValue(), llvm::SUnit::getInstr(), llvm::SDep::getKind(), llvm::TargetInstrInfo::getMemOperandWithOffset(), getPhiRegs(), llvm::MachineOperand::getReg(), llvm::TargetSubtargetInfo::getRegisterInfo(), llvm::MachineFunction::getSubtarget(), llvm::SDep::getSUnit(), llvm::MachineInstr::hasOrderedMemoryRef(), llvm::MachineInstr::hasUnmodeledSideEffects(), llvm::SDep::isArtificial(), llvm::MachineOperand::isIdenticalTo(), llvm::MachineInstr::isPHI(), llvm::MachineInstr::mayLoad(), llvm::MachineInstr::mayStore(), llvm::MachineInstr::memoperands_begin(), llvm::MachinePipeliner::MF, MRI, llvm::SDep::Order, llvm::SDep::Output, SI, std::swap(), SwpPruneLoopCarried, llvm::MachinePipeliner::TII, and TRI.

Referenced by llvm::SMSchedule::computeStart(), and swapAntiDependences().

◆ schedule()

void SwingSchedulerDAG::schedule ( )

overridevirtual

We override the schedule function in ScheduleDAGInstrs to implement the scheduling part of the Swing Modulo Scheduling algorithm.

Implements llvm::ScheduleDAGInstrs.

Definition at line 314 of file MachinePipeliner.cpp.

References llvm::SmallVectorTemplateCommon< T, typename >::begin(), llvm::dbgs(), llvm::Pass::dump(), llvm::SmallVectorTemplateCommon< T, typename >::end(), llvm::Pass::getAnalysis(), llvm::SMSchedule::getMaxStageCount(), I, LLVM_DEBUG, llvm::max(), SwpIgnoreRecMII, SwpMaxMii, and SwpMaxStages.

◆ setMII()

void llvm::SwingSchedulerDAG::setMII ( unsigned mii )

inline

Set the Minimum Initiation Interval for this schedule attempt.

Definition at line 257 of file MachinePipeliner.h.

References MI.

The documentation for this class was generated from the following files:

include/llvm/CodeGen/MachinePipeliner.h
lib/CodeGen/MachinePipeliner.cpp

Public Member Functions

Static Public Member Functions

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ SwingSchedulerDAG()

Member Function Documentation

◆ addMutation()

◆ applyInstrChange()

◆ classof()

◆ finishBlock()

◆ fixupRegisterOverlaps()

◆ getALAP()

◆ getASAP()

◆ getDepth()

◆ getDistance()

◆ getHeight()

◆ getInstrBaseReg()

◆ getMOV()

◆ getZeroLatencyDepth()

◆ getZeroLatencyHeight()

◆ hasNewSchedule()

◆ isBackedge()

◆ isLoopCarriedDep()

◆ schedule()

◆ setMII()